It probably counts pretty much anyone on a newer iPhone/Mac (https://support.apple.com/en-au/guide/iphone/iph00fd3c8c2/io...) and Windows/Bing. Plus all the smaller integrations out there. All of which can be migrated to a new LLM vendor... pretty quickly.
Maybe too much of the same topic? "How R1 was trained" also seemed to quickly fall off. But the big arxiv paper with 1000+ upvotes stuck around a while.
Spot on. I've read the very accessible paper and it's better than any of the how-to's written elsewhere. Nothing against good content being written, but the source material is already pretty good.
> Looks interesting, but I'm skeptical that a book can feasibly stay up to date with the speed of development.
The basic structure of the base models has not really changed since the first GPT launched in 2018. You still have to understand gradient descent, tokenization, embeddings, self-attention, MLPs, supervised fine tuning, RLHF etc for the foreseeable future.
Adding RL based CoT training would be a relatively straightforward addendum to a new edition, and it's an application of long established methods like PPO.
All "generations" of models are presented as revolutionary -- and results-wise they maybe are -- but technically they are usually quite incremental "tweaks" to the previous architecture.
Even more "radical" departures like state space models are closely related to same basic techniques and architectures.
Transformer encoders are not really popular anymore, and all the top LLMs are decoder-only architectures. But encoder models like BERT are used for some tasks.
In any case, self-attention and MLP is the crux of Transformer blocks, be they in the decoder or the encoder.
I have not, but Jay has created a ton of value and knowledge for free and don't fault him for throwing an ad for his book / trying to benefit a bit financially.
For certain file formats, it's true (e.g. gif), but I gotta say- I use "ffmpeg -i input.mov output.mp4" after taking a video on mac, and it looks good and is a tiny fraction (sometimes 100x smaller) of the size.
Same! And I was pleasantly surprised by this working well without any additional parameters.
But I'd also confirm the other comments after going through the steps for shrinking a longer screen recording to <2.5MB with acceptable quality, and cropping some portion of the screen.
I needed a tutorial in addition to the built-in help pages to get it working.
It was a little bit fun almost to try&error my way through combinations of quality and cropping options, but sure, time consuming.
I have to say, I mostly like FFMPEGs approach. Anyone can build anything on top of it, like GUIs.
"Good" defaults can cause an explosion of complexity when providing many different options and allowing all technically feasible combinations.
There's also room for some kind of improved CLI I guess, but many possibilities always mean complex options. So this is probably easier said than done.
It does seem to have pretty good defaults in the MOV MP4 transcoding case.
I think proc gen like this often falls victim to the 1000 bowls of oatmeal problem.
Unless things _feel_ different, it just feels same-y.
Some games handle this well (often through biomes with different rules and/or special areas with different rules) and others (interestingly, often those that talk about how many billions of possibilities there are) are techniquely different... But just all feel the same.
It's interesting to see how game devs continue to be creative in this area and how many games continue to have this problem.
Any game that wants a ton of content has this problem. The many caves in Skyrim start to feel the same after a while. You can often recognize the same assets over and over in very content-rich games. The total number of assets used in the article is tiny compared to what a production game would have. Procedural generation can't be used as an excuse to do less work, or it just feels same-y. A game is still a ton of work; it's just different work for different goals with procedural generation.
Generating a huge world is boring if you don't have interesting objects to put in that world. Interesting objects are meaningless unless they're connected to a story. Minecraft works because it's a sandbox for you to invent your own stories, so the infinite world ends up getting filled by the player. If any element feels repetitive, the whole game will. If you wanted a fully procedural game, you need to figure out how to procedurally generate all the different aspects: story, characters, enemies, places, objects. It's more work to do that well than to make a traditional game. So people going into procedural generation thinking it'll save them work are already destined to fail.
I think Path of Exile does this kind of procedural generation of areas very well. Generated maps of a specific area will feel similar with some landmarks and the direction you're traveling in, but this is a good thing, because they can be used as guide posts where to go if you're an experienced player.
PoE2 on the other hand is some of the worst map generation iv'e ever seen. Like a copy and paste of the same exact thing hundreds of times to give you a large lifeless drudge of a map.
> Procedural generation can't be used as an excuse to do less work
> So people going into procedural generation thinking it'll save them work are already destined to fail
Consider the possibility that nobody really picks up proc-gen in the hopes they can laze out a RDR3 or such over the weekend.
Another thing is, this applies to indies and AAAs alike: while a big world has to have interesting unique things in it, by definition not every square meter can/should be chockful of another "interesting truly unique thing" because if the whole world is filled like that, it's just another kind of sameyness in that the novelty factor would wear out just as quickly once you're getting that there truly is true novelty in absolutely every little square meter, which kills the novelty sensation in a heartbeat. Novelty delights us in a backdrop of routineness, sameyness, same-old-same-old-ness. So in between interesting things, the thusly necessary slightly-"duller" in-between areas are to pace and prep and make one anticipate novelty "hopefully almost just around the corner". Ideally it's so spaced to appear just in time before the player resigns such hopes.
And so if you're going to have slightly-duller "filler areas" (and let me posit that any real-world say forest (in a biome one has traversed before), without the physical air and smells and winds (or friends/pets stringing along) is quickly "proc-gen dull-ish" within minutes — even in reality — or call it "meditative"/calming) — so again, if you're going to have slightly-duller "filler areas" just to connect and space apart the unique content things to good effect, then procedural placements/scatterings/variations are going to beat manual placement not just in "effort time" but because manual would swiftly look much more repetetive (being inevitably eventually effectively copy-paste driven) given the scale of environments under discussion, even if it would not take a human months of menial clickery.
Rockstar gets it right imho but their approach would also get your "many caves in Skyrim" right because they manifest little novel uniquenesses not in scattered objects or env textures/models but via lively interactive interludes, whether it's animals frolicking or chasing each other or attacking towards you or "random stranger" incidents etc. That's the right kind of "filler but not boring", nobody cares about the variety of the rocks, only noticing them if nothing seems to be happening.
The interesting part comes with building a simulation on top of this. For example with simulated citizens that go about their day fulfilling needs, going to work etc.
That's not a game though, and if the citizens all do permutations of the same n-things it's the exact problem that the poster you're replying to is talking about.
Games are interesting because designers have figured out what to present to the player, how to frame it in ways that are engaging/stimulating/challenging/intriguing/etc. That still needs to be authored in some way.
I've periodically seen lightcell and danielle fong in various news / reddit /forums over the last few years and it always seems to be steeped in controversy.
I know next to nothing about the field / tech, but a portion of folks seem to be like "incredible visionary etc. etc." and the another portion like "fringe science / complete bullshit / this is as realistic as cold fusion" kind of thing.
Very interested to hear from folks more in the know of like, high level long term viability / what the implications are etc.
It's a very good idea that is worth pursuing, they are pursuing it. There are many many many problems that need solving between here and "this is a better way to make energy from heat at scale than turning water into steam and spinning a turbine". The science is fundamentally sound but we're nowhere near economic viability.
It's not like cold fusion, the lightcell is based on well-understood physics. The author may be too optimistic with efficiency claim, but those are relatively easy to verify independently.
How do patents work with science actually? I saw upthread that they've patented it, so is independent verification allowed so long as you don't commercially sell it, or give units away at all or so?
I think it looks more than good enough. It loads fast, not bloated and mostly to the point. What's lacking in content is some links to more details. (patents etc.)
She seems like someone with an eye for a clever solution to an existing problem, an eye for funding (her compressed air "LightSail" thing raised over $70 million), and maybe a somewhat shaky relationship with practicality.
reply