This rant about Microsoft controlling TypeScript and through that, re-centralising the whole Javascript ecosystem, is very interesting to me

because

I have always been suspicious of Typescript just because it's *compiled*. It doesn't feel like compilation is native to the interpreted, distributed, Javascript model. It's a single point of trust that doesn't need to be there, a class division that can be exploited.

And welp. Looks like that's exactly what's happening!

alex.kleydints.com/a-post-mort

@natecull While I do hate TypeScript as a language, I need to be careful about glass housing it too much since I write ClojureScript at the day job (and love it, probably the world’s easiest language, it’s even more loosey-goosey, sloppy, TIMTOWTDI than vanilla JavaScript).

I can just gasp in awe at how skillfully Microsoft did this. Longtime Github hater here; it was proprietary even before it was Microsoft, and we had all already been burned twice by Sourceforge and Google Code so I never understood why everyone was so eager to slam their necks into this new guillotine.

@natecull but can’t peeps fork TypeScript if they think it’s so hot?

@natecull never mind; I finished reading the article and the rape joke and the irony of publishing it on actual Medium got a bit too much.

@Sandra My reaction exactly.
I don't agree that the problem is that Typescript is compiled. Because that argument should then go for all compiled languages. There are several entirely open source languages that compile to JavaScript. I would think the issue is that the language and compiler are effectively controlled by Microsoft.

@natecull

@wim_v12e @Sandra

My problem with compilation as compilation is that typically the compiler is built to *run outside of the runtime environment*

and as such it often requires special permission to run, and even specialer permission to modify.

Permissions that increasingly aren't even offered to ordinary users. "No binaries for you, and compilers are hacking tools!" is becoming the norm in corporate environments, and consumer devices are often just as locked down via app stores.

@wim_v12e @Sandra

If the only execution environment a user is offered is a single managed one, then the two processes of:

1. "compilation"

and

2. "compiler development"

MUST be able to take place WITHIN this single managed environment - not outside it - if the user is to have any kind of freedom to modify and trust their personal computational environment.

I keep repeating this because it seems both necessarily true, and yet so often forgotten by developers who have special dev rights.

@wim_v12e @Sandra

Increasingly, those special dev rights (the right to compile/package/publish code, and the even stronger right to modify and develop new compilers and languages) are going to be tied VERY strongly to being vetted members of a coporation, or institution like Github.

The "trusted compiler" is extending softly to become "the trusted organization" and even the open-source world isn't quite noticing that this is happening.

But it was obvious that it would happen from the start.

@natecull @Sandra
I think I need a bit more context to understand this.

If I want to create an android app, I can just do that and put it on f-droid. And I can still create binaries for macos and linux, and last time I checked (about two years ago), I could still build on windows too. I'll need administrator access to install the initial binary, be it compiler, interpreter or VM. But home users have that kind of access on their machines.

So clearly I am missing something here, what is it?

@wim_v12e @natecull @Sandra

This is a sort of parallel dev universe though. Most Android users don't know f-droid exists.

Breaking into vetted spaces gets harder as vetting gets more common for "security" purposes, & then that vetting gets applied in different circumstances. (None of my projects work on iPhone because I'm not paying $90/year for the privilege to be rejected by their app store.)

@enkiv2 @wim_v12e @Sandra

Yep. I have F-Droid on my Android, but I'm pretty bleak in my expectation that it will be around forever.

All it will take is one "industry-wide best practices" update and the next phone I buy won't be able to install F-Droid "because security/privacy risk" or something.

Source: My lived experience this year, seeing Work From Home + ransomware push organizations into MASSIVELY locking down all their machines and "security consultants" whipping them to do it faster.

@enkiv2 @wim_v12e @Sandra

Seriously, before this year I hadn't realised that the tide could turn so swiftly to "literally being able to run a non-corporate-approved EXE is Evil Hacking" but we're here now, it's happening.

We already had Administrator rights removed from all users for years before this. But this year, "running an EXE" is now considered Malware. And running a compiler is doubly so, "because it can allow running non-signed EXEs"

Compilers are generally EXEs.

@enkiv2 @wim_v12e @Sandra

For more awareness of this trend, search for the phrase "living off the land attack"

"Living off the Land" is corporate cybersecurity expert jargon for "being able to access a compiler or even a command line host of any kind on a Windows machine".

And since banning all compilers is now Corporate Cybersecurity Best Practices today, it will be coming to a consumer machine near you tomorrow.

frsecure.com/blog/living-off-t

Follow

@wim_v12e @Sandra @natecull @enkiv2 “Consumer machines” these days are phones and tablets, which are hard to run compilers on in the case of Android and impossible in the case of iCan’tOS.

I suspect most individuals and businesses who want a laptop that locked down will get Chromebooks. Though maybe MS and Apple will try to compete in the Chromebook niche.

@freakazoid @Sandra @natecull @enkiv2

I feel like my original point, that the issue is not if a language is compiled or interpreted, has gotten lost along the line.
What is described in the previous posts is control over execution on the CPU.

The original point made by Nate about TypeScript being compiled is what I responded to. This has nothing to do with permissions to run binaries because TypeScript compiles to JavaScript which is compiled to bytecode for a VM running in the browser.

@wim_v12e @freakazoid @Sandra @natecull

They are related points. An interpreter for a turing-complete language cannot have its expressiveness truncated to a handful of pre-approved flows, so an interpreter available to the user is an escape hatch from the whole signed-binary cage.

@enkiv2 @freakazoid @Sandra @natecull

What we tend to call an interpreter is almost always a compiler combined with a VM, and more often than not the VM incorporates a just-in-time compiler.

You can restrict the definition of "compiler" as "compiler for binaries", but that was not the original point to which I responded. That was about compilation to JavaScript.

@wim_v12e @freakazoid @Sandra @natecull

I'm not sure "almost always" is true. A JIT is pretty tough to write (and until LLVM, if you wanted one you'd have to roll your own) so only quite established languages tended to get one until about 10 years ago. Most interpreted languages, including quite popular ones like perl & Lua, either don't have one or only have a third party JIT that is used by a minority of projects.

@wim_v12e @freakazoid @Sandra @natecull

Anyhow, the context here is how users can be locked out of general purpose computing -- and how a separate compile step supports keeping dev tools out of the hands of users.

Javascript in a browser is a high profile example of a developer tool that users already have installed. It's really the only such example anymore. Moving the Javascript environment to signed bytecode is a good excuse to remove the interactive js console from browsers.

@wim_v12e @freakazoid @Sandra @natecull

The important thing here is that REPL access gives you general purpose computing (whether an interpreter or a JIT is used).

With desktop OSes following in the footsteps of mobile ones (even windows and osx now prefer you get binaries from their pre-approved list in their 'app store' & warning you about unapproved apps at launch, & some Linux distros doing the same) we're headed for a time when writing your own code is way less accessible.

@wim_v12e @freakazoid @Sandra @natecull

Consider the state of console dev for, ex., Nintendo. You pay to apply for access to a dev kit, then pay to apply for official approval to release the finished game, which is manufactured & distributed by the console manufacturer. Copy protection mechanisms also prevent unauthorized releases. The console manufacturer enforces content restrictions, rejects games that compete with their internal dev groups or preferred 3rd partes.

@wim_v12e @freakazoid @Sandra @natecull@mastodon.parties.

Apple has wanted to adopt this model for their PCs since the mid-80s. They *have* for iOS devices. Microsoft is moving toward it too.

Linux distros are pressured to support signing for corporate security & liability reasons.

@enkiv2 @wim_v12e @freakazoid @Sandra
This is repetitive Linux FUD/paranoia, and it is not true.

There are plenty of programming environments on iOS, ranging from Apple's own Shortcuts (née Workflow) and Playgrounds (Swift, so it's slow but it works). Tons of 3rd party programs like Pythonista, Koder, Replete, Hotpaw BASIC; Coda is defunct for financial reasons, but still works if you have it. Or use any of dozens of ssh apps (I use Termius, Prompt is also nice) to run make on a real UNIX.

@mdhughes @wim_v12e @freakazoid @Sandra

Ok, which of these ship (with complete documentation) by default on all iOS devices? Because if they don't, access to them is contingent on the continued good mood of the app store approval team.

@enkiv2 @wim_v12e @freakazoid @Sandra And why not ask someone to come over and install your dev tools for you? Maybe tuck you in and feed you?

If you're competent you can install that stuff yourself, and the user experience is nicer than developing on Android (but so is a root canal).

@mdhughes @wim_v12e @freakazoid @Sandra

On iOS, bypassing the app store entirely?

This is not the sniping argument between Apple fans and Android fans that you seem to want it to be. We're talking about a general trend that affects all popular platforms, and one that Apple's PR department was vital in articulating in the early to mid 80s.

If computers don't come with development environments, then only self-identified programmers learn to code.

@enkiv2 @wim_v12e @freakazoid @Sandra You miss the point entirely. iOS devices have easy access to development apps. Macs have every dev app, Xcode is pushed heavily by Apple. Anyone can easily write iOS software on a Mac, and deploy it themself, doesn't go thru the store.

Your paranoid belief that Apple's taking away "developer freedom" is INSANE. It does not match reality in any way. It is a lie that linux fanboys tell.

@mdhughes > iOS devices have easy access to development apps.

How can this be true when app store policy is that no application that isn't Safari or a skin around it is allowed to download and run arbitrary code? Did I get that wrong?

@wim_v12e @enkiv2 @freakazoid @Sandra

@clacke @wim_v12e @enkiv2 @freakazoid @Sandra That's not been a rule for many years, largely thanks to Pythonista.

You can sync files in Files; it's complicated whether or not Dropbox, git, SFTP, etc. are allowed, but most of the code editors do it. Generally you can download any file and "action" it into an editor.

Frustratingly, Editorial still has no trivial way to import/export except by email or copy-paste encoded data URLs.

@mdhughes Cool! Termux and F-Droid for iOS when?

They'll beat out Android if they go in that direction while Android is going in the lockdown direction.

@wim_v12e @enkiv2 @freakazoid @Sandra

@clacke @wim_v12e @enkiv2 @freakazoid @Sandra There is ish: apps.apple.com/us/app/ish-shel

It's a sandbox, you can't h4xx0r the OS, but works well enough, it's just kind of silly since you can use a good editor instead.

I've used it before to youtube-dl a video and got it out to Files where I could watch it.

@enkiv2 @freakazoid @Sandra @natecull

I said "more often than not", that is not the same as "almost always".

@wim_v12e @freakazoid @Sandra @natecull

It means >50%, which is extremely dubious. There are *maybe* hundreds of languages with JIT support -- a tiny fraction of the total programming language landscape.

It's also dramatically missing the point of the original post to focus on JIT vs traditional interpreter rather than focusing on whether or not end users can edit code.

@enkiv2 @freakazoid @Sandra @natecull

The original poster said

"I have always been suspicious of Typescript just because it's *compiled*."

My reply to that was this:

"I don't agree that the problem is that Typescript is compiled. Because that argument should then go for all compiled languages. There are several entirely open source languages that compile to JavaScript. I would think the issue is that the language and compiler are effectively controlled by Microsoft. "

@wim_v12e @freakazoid @Sandra @natecull

Nate's been posting about these issues on the fediverse for several years now, & that post should be understood in the context of the whole history of discussion that's been happening.

A JIT may *technically* be a compiler, but from the point of view of a developer working in the language, it is an optimization for a language and tool chain that acts as though it's an interpreter.

@wim_v12e @freakazoid @Sandra @natecull

There's a meaningful accessibility distinction to be made between code that takes an additional compilation step and code that doesn't, because code that takes an additional compilation step can be distributed in compiled form without a compiler.

It's perfectly reasonable to assume MS is trying something shady here.

@wim_v12e @freakazoid @Sandra @enkiv2

Thanks for your thoughtful response!

Compilation/transpilation to Javascript, I think, is still quite problematic for me.

If the compiler is written in Javascript and compilation can be done in small pieces and from within the Javascript runtime environment, then at least it doesn't have the "you don't have any control over the compiler" problem we're increasingly facing.

But I'm still not a huge fan because you lose information when you compile.

@wim_v12e @freakazoid @Sandra @enkiv2

We went though this whole thing decades ago with "fourth-generation languages" and preprocessors. Source code generated by machine is not human-friendly at all.

This could probably be mitigated IF someone constructed a whole interactive execution environment within Javascript where, say, you had objects that had both Typescript (or whatever input langage) source code and then the compiled-to-Javascript object code.

I'm sure we could build this on WASM.

@wim_v12e @freakazoid @Sandra @enkiv2

But although this seems obvious to me (it's only "obvious" because I'm thinking as a user who grew up in BASIC, who expects to always have an "immediate mode" - with data/code save/load facility - available inside a runtime environment and I'm always shocked that there often isn't) it doesn't seem to be the way that many of these compile-to-Javascript toolchains think.

All source code is generally kept in files.

But browsers often don't give you files.

@wim_v12e @freakazoid @Sandra @enkiv2

This MIGHT be about to change / in process of changing now that the File System Access API has been approved and is in the most recent Firefox (not yet in long-term support but I think next month).

Anyway, what I want is for the whole "source, version control, build, compile, verifiy, package, sign" toolchain to be available at runtime, for every user, within every deployed endpoint.

Generally I think that means the toolchain should be small not big.

@wim_v12e @freakazoid @Sandra @enkiv2

Some form of "signing" or secure attestation might well be an important basic primitive to have in any distributed code execution environment. Especially if users are going to construct things like "types" at runtime and send them over a wire to untrusted distant machines. That's a thing I think we need so we might as well assume that we need it.

But that kind of signing is gonna have to not be based on X.509 certificates and PKI.

@natecull @Sandra @enkiv2 @wim_v12e Authorizing code execution based on someone else attesting to its safety does not scale well. We need object capability security.

@freakazoid @Sandra @enkiv2 @wim_v12e

Yep, object capabilities does seem like a good way forward. Maybe ocaps don't actually need public key cryptography, though I think mutable objects like in Goblins probably do. That's the sort of 'signing' I mean. It needs to be small and fast and not involve a third party, and you probably wouldn't even use it locally, within your own trusted hard drive or LAN.

(I'm sure to what extent we can trust hard drives and LANs, especially removable media).

@freakazoid @Sandra @enkiv2 @wim_v12e

I would love for us to have some kind of universal "trusted persistent storage" where you could store very simple object-type structures where each object is guaranteed to meet some contract/type/criteria/function. Preferably in as simple and general a way as possible.

Filesystems don't get us this. Databases sort of do but they're chunky and very hard to update the schema. "Object databases" tried to do this but failed - I think embedded too much code.

@freakazoid @Sandra @enkiv2 @wim_v12e

Some people are advocating SQLite for this kind of personal data store. I still think SQL is too slow and chunky for this, but maybe it's okay.

But whatever we used, it would have to deal with "what happens if someone imports a foreign disk / SD card / zipfile / SQLite file that's been constructed/edited by an adversary so the attestations it makes about contracts/schemas/types are deliberately evil and broken, how do we validate it fast and safely?"

@natecull
Interesting thread, bookmarked,

I don't understand exactly what you are trying to do, but:
1. We've used sqlite a lot and it has been pretty bulletproof.
2. Depending on what you want to validate, some structure to the data (which a relational database gives you) can help.
3. It's not the only way to structure the data, though.

@freakazoid @Sandra @enkiv2 @wim_v12e

@bhaugen @freakazoid @Sandra @enkiv2 @wim_v12e

The sort of thing I'd like would be something like

"what if an interactive Javascript interpreter where every object gets persisted to a file"

Seems like trying to persist arbitrary binary objects to a SQL database might not be the best fit.

@natecull
"Seems like trying to persist arbitrary binary objects to a SQL database might not be the best fit."

Nope. But maybe a Json DB? Those seem to be popping up like flies. Have no idea which are any good, though.

@freakazoid @Sandra @enkiv2 @wim_v12e

@bhaugen @natecull @freakazoid @Sandra @wim_v12e

Tbh, RDBMSes are not a great fit for persisting arbitrary objects. A prototype-object system like Javascript has would be an even worse fit. ODBC systems tend to provide the worst of both worlds, even for languages where you expect a handful of selected static classes to be persisted. To persist an entire JS environment, you'd probably also want history (since rolling back state gives you constraint programming & capabilities security free)

@bhaugen @natecull @freakazoid @Sandra @wim_v12e

(Or 90% of it anyway, if you do it right)

So you'd want to roll your own that lets you fragment out shared keys and shared values with references to other objects or the same object at different times while being fast to index & very small -- which an RDBMS won't do by itself for objects & can't be reliably made to do.

Naturally this means rewriting the JS implementation entirely too. (And breaking compatibility to add those features you want)

@enkiv2 @wim_v12e @bhaugen @Sandra @mathew @natecull Yeah I don’t know where this whole notion of SQL being slow and clunky or bloated comes from. SQLite, PostgreSQL, and MySQL/MariaDB have all had engineer-centuries put into them, and they’re rock solid, high-performance databases. They’re incredibly hard to beat. Most “schemaless” databases are extremely immature by comparison.

@bhaugen @wim_v12e @Sandra @mathew @natecull @enkiv2 I should say they’re incredibly hard to beat in any practical application. The NoSQL snake oil salesmen love to show how they beat them on narrow benchmarks or specific applications. But even if you have an application for which some other DB is faster, what about tooling? How do you back up? Can you take a consistent snapshot? How do you restore?

@bhaugen @mathew @natecull @enkiv2 @Sandra @wim_v12e For schema migration, your best bet is usually to stick an application-specific API in front of the database both to centralize all your data access code (including caching) and handle everything related to migration, including double-writing and falling back when a record isn’t in the new table yet.

@freakazoid @bhaugen @mathew @natecull @Sandra @wim_v12e

Right, but if you are storing arbitrary objects, you literally don't have a schema. (Or, your "schema" is so granular that the DB is useless.)

Most real-world applications that use nosql would be better off using a properly designed RDBMS, no doubt. The use case here -- to provide MUMPS-style persistence of arbitrary Javascript objects (including code) within every interpreter cycle -- is not a good fit for RDBMS or any existing nosqldb

@wim_v12e @Sandra @bhaugen @mathew @enkiv2 @natecull Then the question becomes what it means to store “arbitrary objects”. Why are you doing it and what do you expect the database to do with their structure? Do you expect indexed search on every field an object might have? What do you plan to do with that?

OpenStreetMap’s tags are kinda like this, but they work fine with SQL.

Show newer
Show newer

@freakazoid virtually every “nosql” database I’ve used in a production environment winds-up reimplementing SQL & RDBMS features*, and in all cases the results are worse than a mature RDBMS.

That’s not to say I don’t love k:v stores, object warehouses, etc. these things are awesome when used for the right applications.

Where it goes to shit is when something needs a database and the implement it’s choose nosql not because they make a conscious compromise on features, but when they don’t understand the problem sufficiently (or just don’t want to) to design the database properly from the beginning.

(* Redis may be the only exception to this, which is why it’s my favorite.)

@bhaugen @wim_v12e @Sandra @mathew @natecull @enkiv2

@requiem @freakazoid @bhaugen @wim_v12e @Sandra @mathew @natecull @enkiv2

You have to remember that originally it was The Company Database which ran on The Company Mainframe. At google scale that didn't work, hence NoSQL.

But since then this whole ecosystem has grown up around selling developers this dream, while in fact the value they're providing is making it so app devs don't need to learn SQL.

@requiem @freakazoid @bhaugen @wim_v12e @Sandra @mathew @natecull @enkiv2
That came across a lot more authoritative than intended, insert IMOs and "it seems like"s appropriately...

@cjd The NoSQL name is funny. SQL, while a language with many flaws both in itself and in the culture around it, wasn't the main point of NoSQL.
What didn't work at Google scale was first and foremost the consistency guarantees, and after that maybe the schema.

@natecull @mathew @wim_v12e @enkiv2 @bhaugen @freakazoid @requiem @Sandra
Show newer

@requiem @freakazoid @bhaugen @wim_v12e @Sandra @mathew @natecull @enkiv2 what do you think about PostgreSQL with its JSON field type? Seems to me that it'd be the best of both worlds?

Show newer

@freakazoid @enkiv2 @wim_v12e @bhaugen @Sandra @mathew

'I don’t know where this whole notion of SQL being slow and clunky or bloated comes from. "

By "chunky" I don't *necessarily* mean "slow" - but rather that SQL is quite an extremely high-ceremony wrapper around "creating or looking up an object", if you wanted to use it to, eg, persist in-memory objects of unknown shape.

In Lisp, an object creation is CONS and an object lookup is CAR/CDR.

SQL is *quite a bit* chunkier than that, no?

@freakazoid @enkiv2 @wim_v12e @bhaugen @Sandra @mathew

Also, there's no direct connection (except in some exotic very platform-specific frameworks, like I think .NET has) between defining a type in a programming language and defining a schema in a SQL database.

Despite types and schemas both being ways of describing the "shape" of objects.

It's this amount of high ceremony, where you have to commit to a schema before you store data, that makes SQL not a good fit for exploratory programming.

@freakazoid @enkiv2 @wim_v12e @bhaugen @Sandra @mathew

IF you've got a LARGE AMOUNT of EXTREMELY REGULARLY SHAPED objects, and you need the accesses to be fast, and you aren't going to be moving them between systems very much, and you're comfortable with there being quite an involved schema-definition / table-creation process that might not be very easy to just pick up and move elsewhere...

... then sure, SQL, okay.

I *guess* you could maybe use it for bulk save/load of in-RAM objects?

@freakazoid @enkiv2 @wim_v12e @bhaugen @Sandra @mathew

But (with the possible exception of SQLite) I kinda distrust SQL databases in the "app running on my personal desktop/phone" sense, because a facility I REALLY REALLY want there is the ability to, in an emergency situation and under time pressure, just be able to instantly copy off "my data" to another machine and bam, instantly be up and running, zero data loss.

Migrating data with zero loss is traditionally quite dangerous in SQL.

Show newer

@natecull @freakazoid @enkiv2 @wim_v12e @bhaugen @Sandra @mathew Yeah, I agree!

SQL is an amazing technology, but few projects find they really requires it's power. Leaving them complaining about working around that power.

Sign in to participate in the conversation
R E T R O  S O C I A L

A social network for the 19A0s.