> It’s a grim week for Meta. The company formerly known as Facebook, and before that Facemash, “designed to evaluate the attractiveness of female Harvard students,”

Wow if that's the opener, I expect the rest to be SUPER emotionally charged

Show replies

As a long-time sailor, this case may have no impact on regular folks, but I maintain a fool's hope a Meta victory will significantly weaken the copyright system.

Show replies

I stole a lot of books too, reading them and all. Just integrated them into my worldview, and don't pay a license fee when I use the ideas in new contexts. Sometimes I even quote from them. A lot of them I didn't even pay for, I borrowed them from libraries or friends.

Show replies

Ok? The Google lawsuit and promising lawsuit of the Open library probably will result in a W for Meta. Torrenting is obviously the best way to grab lots of data to train on. Just because they seeded (torrent clients automatically do this) doesn't mean they actually uploaded anything if they couldn't connect to a peer or manually paused/stoped the torrent. Also the author slants the story against Meta and has a bias. At least I felt that way when reading it.

Show replies

My first (almost) thought was "Ok, looking at the llama, where's the quality boost?" I think the explanation is probably in how LLM are trained. Even without knowing deeply the internals, I suspect that it's the compression of information so to simplify you can't make 8GB data contain all the facts of a "bigger" normalized relational database. So they keep the facts present everywhere and often drop rare facts. For example, a fact "SQL was invented at IBM", this fact can be found everywhere, in books, web sites, comments. You don't need access to copyrighted books to acquire this fact. But a first-person account of someone who worked at IBM at that time is probably can be found in a couple of books, but due to "compression", it will be gone anyway