If you expect Jena to be more battle-tested because it is older, forget it, if the process is killed by a unexpected shutdown or some other reason it results in data corruption. At least this was my experience a few years ago.
I found graph databases a beguiling idea when I first learned about them, and this is a welcome addition, but I've since temperated my excitement. They are not as flexible and universal a modal as is often promised. Everything is a graph, sure but the result of your SPARQL query not necessarily.
I found classical DBMS based on sets/multisets to be much easier to compose from a querying point of view. A table is a set/multiset and a result of a query is also a set/multiset, SPARQL guarantees no such composability. Maybe, if you want to start mucking around with inference engines, but you'll either run into problems of undecidability.
Jena lets you make little in-memory triple stores that you can use the way people use the list-map-scalar trinity. I've been working on this publication about that (RDF for difficult cases and when ordering counts) for years and it just got published last week
I'll call out my collabortor Liju Fan for being the only person I've met who knew how to do anything interesting with OWL. (Well, I can do interesting things now but I owe it all to her.)
(For the research for that paper I used rdflib under PyPi because CPython was not fast enough.)
When I needed big persistent triple stores (that you use the way you might use postgres) I used to use
and had pretty good luck if I loaded a billion triples if I used plenty of 'stabilizers' (create a new AWS instance with ample RAM, use scripts to load a billion triples starting from an empty database, shut it down, make an AMI, start a new instance with the AMI, expect it to warm up for 20 minutes or so before query performance is good)
I don't regularly build systems on SPARQL today because of problems with updating. In particular, SQL has an idea of a "record" which is a row in a table, document oriented databases have an idea of a "record" which is a bit more flexible. Updating a SPARQL database is a little bit dangerous because there is no intrinsic idea of what a record is; i mean, you can define one by starting at a particular URI and traversing to the right across blank nodes and saying it is a 'record' and it works OK. But it's a discipline that I impose on it with my libraries, it ought to be baked into standards, baked into the databases, wrapped up in transactions, etc. For anything OLTP-ish I am still using SQL or document-oriented databases, but I hate the lack of namespaces and similar affordances that make SPARQL scalable in terms of "smash together a bunch of data from different sources" in document-oriented databases wheras SPARQL is missing the affordances you have in document-oriented databases for handling ordered collections. We badly need a SPARQL 2 which makes the kind of work that I talk about in that technical report easy.
> Updating a SPARQL database is a little bit dangerous because there is no intrinsic idea of what a record is
SPARQL has a notion of a transactional boundary just like SQL has. You can combine multiple SPARQL queries in one transaction, they will all succeed or all fail just like you'd expect.
Your code has to put the right things in a transaction all the time for transactions for transactions to work right. If there is some flow of information like
application does query -> application thinks -> application does update
you have to wrap the whole sandwich in a transaction, people frequently don't do that. If I'm writing 20 of those for an application I want something that I know is bulletproof.
My experience with SQL is that the average SQL developer doesn't really understand how to do transactions right but their ass gets saved (in a probabilistic sense) by the grouping of updates that is implicit by running an INSERT or an UPDATE against a table.
There's also the fact that a lot of triple stores are seriously half baked research-quality code if that. Many triple stores struggle if you just try to load 100,000 triples sequentially, for an application like my YOShInOn RSS reader which I expect to use every day and not have to patch or maintain anything for 18+ months. (Ok, a 20GB database that needs to be pruned crept up on me gradually, but that's an arangodb problem, I'd expect the average triple to store to have crumbled 17 months ago.)
I'd love to have something that updates like a document-oriented database but lets you run a SPARQL query against the union of all the documents. Database experts though always seem to change the subject when it comes to having a graph algebra that lets you UNION 10 million graphs.
(For that matter, I sure as hell couldn't pitch any kind any kind of "boxes-and-lines" query tool [1] etc. that passed JSON documents/RDF graphs over the lines between the operators to the VCs and private equity people who were buying up query engines circa 2015 because they were hung up on the speed of columnar query engines... Despite the fact that the ones that pass relational rows over the lines require people who really aren't qualified to do so create analysis jobs that look like terrible hairballs because of all the joins they do.)
> you have to wrap the whole sandwich in a transaction
True, SPARQL does not allow "opening" transactions such that you can run one query, do some logic, and run another query while doing commit. Which was a pain for me. RDF4J has a non-standard API to do that, I think they are trying to upstream it to SPARQL 1.2.
> There's also the fact that a lot of triple stores are seriously half baked research-quality code if that.
Also true. Although excellent researchers who wrote one of the best reasoners (Pellet) decided to leave academia and make a production grade system. They succeeded with Stardog but you don't want to know how much a license costs.
> couldn't pitch any kind any kind of "boxes-and-lines" query tool [1] etc. that passed JSON documents/RDF graphs
I really enjoy this talk from one of the creators of OWL [1]. There, he makes a point that OWL is unpopular not because it's too complex but because it's not advanced enough to solve real problems people care about (read: ready to pay money for). I think the case you described involves VCs having clarity on how to make money off one thing but not the other. I do think that the Semantic Web 3.0 (if we count Linked Data as a Semantic Web 2.0 aka Semantic Web Lite attempt) will need a better (appealing to business) case than the one presented in the 2001 SciAm paper.
OWL ontologies making a big comeback as part of Knowledge Graph groundings for LLM outputs. And several SPARQL and RDF knowledge graph startups are VC-baked and thriving. The world is a big place.
Personally I thought Stardog was trash, but if I'd had different requirements I might be happy with it.
The trouble w/ OWL as I see it (talked about in that TR) is that people don't really want "first order logic", but they want "first order logic + arithmetic" which is a nightmare that Kurt Godel warned you about. (That ISO 20022 which that TR is related to is about the financial domain which is all about arithmetic)
After Doug Lenat's death a lot of stuff came out that revealed the problems w/ Cyc, not least that even if you try to build something that is "knowledge based" it can't practically solve all the problems you want using a SMT-based strategy but you have to build a library of special purpose algorithms for everything you want to do and it turns out to be a godawful mess.
I'm disappointed that the semweb community hasn't made a serious crack at usable and efficient production rules (dealing w/ problems like negation, controlling execution order, RETE execution, retraction) instead we get half-answers like SPIN with fixed-point execution (used an even more half-baked version of that to research that TR, gets you somewhere). Of course, production rules never got standardized in any domain because nobody can agree on the way to address those four issues even though it usually isn't hard to find an answer that's fine for a particular application.
(It's a frequently problem that experts on a technology can get by on half-baked specific answers that would need a general solution if they were going to be useful for a general audience. One reason why parser generators are so bad is that if you understand parser generators enough to write a parser generator you aren't bothered by the terrible developer experience of parser generators.)
Sorry for the negativity Kendall but the semweb didn't return the love that I gave it. I did hundreds of sales calls that went nowhere, but my phone kept ringing for people who wanted me to work on neural nets.
That’s tough. Not sure what that has to do with Stardog. Biggest companies in the world rely on it daily and you say it’s trash. I couldn’t find an email from you using it since 2013. I guess we figured something out. NNs are cool too; at last count we use half a dozen different ones including GNNs… NeSy is hot and I can hardly read a paper these days that doesn’t talk about triples.
(1) I'll grant it was a long time ago. Things could have changed a lot.
(2) It's generic that a new database comes out, gets hyped, but turns out to be "trash" when you try to use it. If a new database was actually good that would be exceptional. (Probably in 2013 it satisfied somebody's requirements but the hype for Stardog in 2013 seemed to be entirely out of line with what I needed for the project I was doing at the time)
I thought Postgres was trash in 2001 and called it CrashGreSlow, now I swear by it. Early on people were making big claims for it that were not substantiated but people did the hard work over a long time to make it great.
I thought mongodb was trash when it came out, then I worked for a place that used it despite the engineers believing it was trash and begging me not to use it for a spike prototype. It never got better. Now it is common knowledge that mongodb is trash.
(3) Maybe it's not fair but I was hurt by the experience, my wife was furious at the balance I'd run up on the HELOC chasing my Moby Dick. As an applications programmer who was accustomed to getting things right I had a terrible opinion of most of the luminaries in the semantic web field at the time many of whom were shipping code that was academic quality at best.
datomic (and partially xtdb /former crux) are OLTPish, and use only such "tuples" , essentially it's up to the user to define what constitutes an entity if at all ("row", "object", "document", whatever) - maybe some entity-id and everything linked to it, but maybe other less-identity-related stuff. Which might feel freeing to extent, but as you said, also expects great responsibility/discipline to cobble the proper properties together.
which could be written as SPARQL queries, I've used these to cut records out of a big graph, I haven't thought seriously if these could be built into a large scale general purpose systems.
The most fun I ever had with Jena was when I used the rules engine for the control plane of a batch processing system which
used stream processing primitives [1]
The Jena folks said my use was completely unsupported, I had looked at the source code and got to understand how the rules engine worked and I knew damn well there was nothing wrong with what I was doing.
I've thought a lot about why production rules have had so little impact on the industry, I mean people really hate drools
That kind of system is particularly strong at handling deep asynchrony, like when a business process at a bank might involve some steps where you might have to wait for a loan office to approve a loan. It's disappointing to me that nobody has tried to use them (so far as I can tell) to deal with the asynchronous comms problems in Javascript though I've yet to get a clear picture in my mind about how to get started on that. (Funny I am getting an idea now so I'm putting a ticket on my personal Kanban board)
[1] I worked later at a place that had a similar engine written in very awkward Scala that allegedly used Either and Optional for error handling but actually dropped errors most of the time; I knew what algebra my engine supported, they argued whether or not something like that had an algebra; my engine got the same answers every time because it tore down the system properly at the end, their engine gave different answers every time but they didn't seem to care
I said suitable for newcomers aka people touching RDF for the first time. If you want production-ready, you probably want Stardog, Ontotext GraphDB, or AWS Neptune - neither is cheap. https://github.com/the-qa-company/qEndpoint is also an interesting project that's used in production.
With Neo4j properties and list members cannot be complex objects just like DB table rows. I was thinking that my dream DB would be a hybrid of MongoDB documents with Neo4j type relationships between them.
I understand, but what if I want to use the enums the way they are used in C, as a label for a number, probably as a way to encode some type or another. Sum types of literal numbers are not very practical here because the labels should be part of the API.
That doesn't give you a type that you can use for the actual enum values. If you wanted a function argument to take in one of your enum values, you'd still have to use keyof in the signature like:
function doSomethingWithMyEnum(val: MyEnum[keyof MyEnum])
You could do `val: number`, but now you're allowing any number at all.
Ultimately, the type syntax in TypeScript is a key part of the language, and I don't think it's unreasonable to expect developers to learn the basic typeof and keyof operators. If we were talking about something wonkier like mapped types or conditional types, sure, it might make sense to avoid those for something as basic as enums.
This is way harder to parse and understand than the enum alternative.
Personally I am definitely not skilled enough at typescript to come up with this on my own before seeing this thread so this was not even an option until now.
You get used to it! Also much easier to read in an editor with intellisense than your first exposure as plain text. If you're going to spend any considerable amount of time writing typescript, parent's advice is good.
Well, that's basically how you would have done it in vanilla JS before typescript came around. The main awkwardness is the type definition. I often prefer to use a library like type-fest for this kind of thing so you can just say:
export type MyEnum = ValueOf<MyEnumMapping>;
TypeScript not having enough sugar in its built-in utility types is definitely a fair criticism.
But more to the point, the above is not usually how you do enums in TS unless you have some very specific reason to want your values to be numbers at all times. There are some cases like that, but usually you would just let the values be strings, and map them to numbers on demand if that's actually required (e.g. for a specific serialization format).
No, this is what I suggest for someone who wants to do something non-idiomatic because they're used to C. What I suggest in most cases is just a string union type.
Edit: But for what it's worth, yes, the above is still more elegant than enums. The syntax may feel less elegant, but among other things, the above does not depart from the structural type paradigm that the rest of the language uses, all for something as simple as an enum.
How is this different from the Durable Promises idea by ResonateHQ?
It seams to me a bitter easer to get started with as a small standalone project.
Would you mind comparing this solutions to that?
I skimmed a bit about Durable Promises, and if I understand correctly, they’re similar to workflow tools like Temporal but come with high-quality language bindings that let you write workflow steps using async/await.
We built something almost identical in Rust to let us use async/await for long-running & failure-prone workflows. This powers almost everything we do at Rivet. We have a technical writeup coming later, but here's a couple links of interest:
Rivet Actors can start, stop, or crash at any time and still continue functioning, much like Durable Promises.
However, their scope differs: Rivet Actors are broader and designed for anything stateful & realtime, while Durable Promises seem focused on workflows with complex control flow.
Rivet Actors can (and likely will) support workflow-like applications in the future, since state management and rescheduling are already built-in.
Maybe homology can help, it is a sort of calculus for discrete structures where you count how many N dimensional hole there are over time. Dunno about NN but that is what they can do with fMRI.
There is also a difference between "bible study" and "historical criticism", where the later is devoid of any specific religious interpretation.
I am secular person and I find it interesting how ideas evolve over time. I think it was around the year 1000 that Jewish scholars started to wonder why the old testament didn't mention the planets which where a Greek discovery and cultural "meme". People where worried about things like that, but couldn't formulate a good answer. Also most people don't know but ancient Judaism was a polytheistic religion, and only became monotheistic after the return from the Babylonian exile.
UV damage to internal tissues seems unlikely given that the tartrazine dye they used absorbs strongly in the UV region of the spectrum. You can see this in Figure S1 A & B:
Also the abstract of the article notes that strong UV absorption is likely a prerequisite for this effect:
> We hypothesized that strongly absorbing molecules can achieve optical transparency in live biological tissues. By applying the Lorentz oscillator model for the dielectric properties of tissue components and absorbing molecules, we predicted that dye molecules with sharp absorption resonances in the near-ultraviolet spectrum (300 to 400 nm) and blue region of the visible spectrum (400 to 500 nm) are effective in raising the real part of the refractive index of the aqueous medium at longer wavelengths when dissolved in water, which is in agreement with the Kramers-Kronig relations. As a result, water-soluble dyes can effectively reduce the RI contrast between water and lipids, leading to optical transparency of live biological tissues.
However this kind of research into the effects of absorption bands on the transmission properties at interfaces might ultimately bring about more effective sunscreen formulations.
> UV damage to internal tissues seems unlikely given that the tartrazine dye they used absorbs strongly in the UV region of the spectrum
To expand: "the most hazardous UV radiation has wavelengths between 240 nm and 300 nm" [2]. While tartrazine has a lambda max at 425 nm in water [2], it has a second ridiculously-convenient peak around 260 nm [3].
TL; DR It should be mildly UV protective ceteris paribus.
like an x-ray, I'd risk that for a one-off doctors appointment, but I'd probably not risk it on my body at all times. maybe there are safer dyes that have the same effect
I found graph databases a beguiling idea when I first learned about them, and this is a welcome addition, but I've since temperated my excitement. They are not as flexible and universal a modal as is often promised. Everything is a graph, sure but the result of your SPARQL query not necessarily.
I found classical DBMS based on sets/multisets to be much easier to compose from a querying point of view. A table is a set/multiset and a result of a query is also a set/multiset, SPARQL guarantees no such composability. Maybe, if you want to start mucking around with inference engines, but you'll either run into problems of undecidability.
reply