Translating an app's data design into a database can silently drop its rules; data built to break each rule shows which. A rule here is a promise about the stored data, such as "each conversation is saved once".
Picture a hobby-group app. It saves facts like "Ana and Ben talked inside the Climbing group." Its rules say this conversation is saved once, and that Ana–Ben is the same conversation as Ben–Ana. The same design was translated into three databases: PostgreSQL, MongoDB and TypeDB. The first two are widely used; TypeDB stores data as links between things. Asked to save the conversation twice, PostgreSQL and MongoDB refused. TypeDB saved it, with no error. An app counting Ana's conversations would now count two, and nothing would say why.
The design and the usual way to use it
The written design of what an app stores, and the rules that data must obey, is called a data model. Here a conversation joins two users and one group. Each user has one email and an id, a code number that names exactly that user.
Each database needs its own setup: where each kind of data goes, and the checks it runs on every save. For PostgreSQL, the setup includes a table, a grid named conversation, with a line holding Ana's id, Ben's id and Climbing's id.
Usually a tool translates the data model into each setup. The translation keeps the shape, so Ana, Ben and Climbing all have a place and it looks right. Whether each rule came along stays invisible until bad data arrives.
Testing the translation instead of trusting it
factgraph is a public tool that translates a data model and then tests the translation. It lists every rule, one promise at a time: 10 here. For each, it builds a small set of data that breaks it. Where possible, it sends that data to a live copy of each database and records whether it was refused.
Three of the 10 rules are below: the two that TypeDB lost, and one that all three kept. All three also kept the other seven. Pick one and see what each database did.
Why TypeDB kept the duplicate
In PostgreSQL, the setup says no two lines of the conversation table may be identical. It also accepts the two people in one fixed order only, so Ben–Ana cannot slip in as a second line. MongoDB stores each conversation as a document, a small labelled bundle of values, under the same two rules.
TypeDB stores the conversation as a link between Ana, Ben and Climbing, with a named slot for each person: first and second. Nothing in the setup factgraph wrote says the two slots are interchangeable, or that a link may exist only once. So it stored the copy. factgraph's translation dropped the rule, not TypeDB.
The reverse also happens. PostgreSQL and MongoDB have one place for Ana's email, so a second cannot be written. TypeDB keeps emails as separate values, so it needs a rule against a second. It had that rule, and it refused.
All 10 rules
Seen live means a running database was sent rule-breaking data and its answer recorded. Claimed means factgraph judged the answer from the setup it wrote, without sending data. A second email in PostgreSQL is claimed: there is one place for an email, so nothing to send.
| Rule | PostgreSQL | MongoDB | TypeDB |
|---|---|---|---|
| Ana–Ben is saved once | refused (seen live) | refused (seen live) | stored the copy (seen live) |
| Ana–Ben equals Ben–Ana | refused (seen live) | refused (seen live) | not kept (claimed; live test unclear) |
| Ana, Ben and Climbing have required values, and each id names one of them (5 rules) | kept (seen live) | kept (seen live) | kept (seen live) |
| Ana has at most one email and one id, and Climbing one id (3 rules) | kept (claimed) | kept (claimed) | kept (seen live) |
The difference between seen live and claimed matters. The first live runs exposed three mistakes in factgraph itself, now fixed and listed in its list of changes. In one, its PostgreSQL setup did not even load, because a table named User clashes with a reserved word. A reserved word is one PostgreSQL keeps for its own commands. Its claimed results had not caught that.
When to use this, and when not
Use this test when one data model feeds more than one database, or when a database's rules were an afterthought. Skip it for a single database whose rules someone writes and tests by hand. It is not a speed test: it sends a few records, not millions.
Limits
- Only one small data model was tested.
- The databases were throwaway test copies.
- TypeDB's Ben–Ana test gave no clear answer, because the test users also broke another rule.
- Whether TypeDB could keep these rules with a different setup was not tested.
- factgraph's output for GraphQL, a way for apps to request data, is untested; GraphQL stores nothing.
The code is public as factgraph-public.