“Change Is Reversible transformation is not” – Ezgi Karaboya
As a learning exercise, I am writing my own programming language. Its working name is Goblin.
I am developing Goblin as a safety-conscious language that combines ideas from scripting languages and databases, with the aim of creating a flexible, all-in-one environment.
This week, I added the ability for every part of a program to reverse-engineer itself back into Goblin's own syntax. It took a few attempts and plenty of testing, but you can now simply type write 'filename' and the database will reconstruct itself as the same code used at the interface level.
This means you can save scripts partway through execution and let Goblin handle the rest. You can also run simulations, save them midway, and later resume from exactly where you left off. This approach is slightly less compact—and potentially slower—than the way SQL stores records. SQL relies more heavily on direct disk-level storage, with rows stored in tables using predefined field sizes. For example, when creating a table, you specify the maximum number of characters a string can contain using types such as VARCHAR or NVARCHAR. This allows the database to determine file and column sizes in advance, making it easier to deserialize data from the file.
Goblin, however, uses nodes that behave like dictionaries. Unless I can reliably fix their size and prevent it from changing, they are less suited to this kind of predefined storage. Fixing their size would also limit Goblin's future usefulness as a scripting language, so I am working to find a practical middle ground.
A visual You Tube walkthrough is available here: I Wrote A Programming Language - Reversible Language
Reversible Programming Language
This led me to the idea that Goblin should be a reversible programming language: every atomic component of the code understands how to convert its internal data structure back into basic Goblin syntax.
This capability is built into every part of the language. A node, function, or group contains its own documentation and can therefore write itself to disk at any time. In many cases, this also includes comments for functions and groups that explain the logic behind a file's purpose.
Goblin goes beyond languages such as SQL and Python, where a developer's script is stored separately. The code below is written and deployed entirely by Goblin itself, and it describes how to recreate its current state.
Therefore when you write the above is what ends up in the file.
Scripting Files
These files can be opened and edited like any other script files, then run as commands in the terminal. This is broadly how Python works, although I also need to store data within the files themselves.
I believe this approach is sufficient because it preserves all data. That said, I acknowledge that it is somewhat more write-intensive than SQL. To address this, I developed normalise and de-normalise commands, which reversibly transform the way data is stored between different states or configurations, maximising its usefulness for the task at hand.
Normalise and De-normalise as commands
I have long found many conventional approaches to normalisation in SQL and other database systems somewhat counterintuitive. Although normalising tables can reduce disk usage, it often adds computational overhead: additional joins require the system to scan storage and match keys in memory.
In a Goblin database, a denormalised record contains no duplicated data—only pointers to the locations where that data can be retrieved. This makes duplication impossible, even at the smallest scale, while ensuring that RAM is not wasted unnecessarily.
In SQL, performance bottlenecks often arise when queries depend on unindexed keys, placing strain on both disk I/O and CPU resources. As more tables are joined, computational complexity can grow rapidly and non-linearly. Goblin addresses this by keeping more data in memory and using a similarly denormalised structure, but without the join overhead or escalating computational costs commonly associated with SQL. This makes it particularly well suited to a dedicated language for building analytical sandboxes.
Organised in this way, the system begins to resemble a graph database while retaining the behaviour of a relational database. Normalised fields can also be grouped into dedicated collections to improve access times and provide columnar-database-like behaviour. These collections serve a role similar to SQL indexes, acting as both bookmarks and structural guides within the data.
Because the code is self-aware and reversible, these groups can be recreated cleanly when data is written back to files and later reconstituted. This introduces the idea of an explicit access-performance strategy. Although SQL can apply indexes through a command, their effects are often opaque: a poorly optimised join may consume excessive RAM, while users may not realise that a query depends on unindexed keys. By requiring users to participate in building and invoking the infrastructure that governs access speed, Goblin encourages a more deliberate and responsible approach to performance.
Traditionally, building these databases required gathering all the data, creating a logical model, and following a step-by-step process. After creating several tables, you would normalise the data by separating repeated strings, dates, and numbers into different tables. Because a key might use only one or two bytes while a string uses one byte per character, this could significantly reduce file sizes.
However, doing this effectively has always depended on the user understanding the data model. In my experience, people often leave data in a highly normalised state even when using it for reports, where speed is essential. That is why I added simple commands to Goblin that allow a dataset to be normalised or denormalised.
This can be taken to an extreme. In the example above, the normalise command moves all data into Sixth Normal Form (6NF), compacting files as much as possible. Because the connections are stored as in-memory graph relationships, the performance cost of looking them up is negligible. At present, you simply need to know that the data has been configured this way and that some of its memory footprint has been offloaded into relationships.
In the code above, the initial create commands create nodes whose data is primarily stored in relationships rather than directly within the nodes themselves. This reduces RAM usage. It also goes further than traditional SQL can practically achieve, because SQL joins generally rely on indexes for optimisation. Each indexed key consumes memory, meaning you can quickly reach a practical limit.
When you need to perform frequent reads and writes on a small number of nodes, you can call denormalise on them. This increases their RAM footprint but reduces access time. You can later normalise them again, or choose to keep particular fields normalised while leaving others denormalised, depending on your access-performance strategy.
Transformation Is Non Reversible
“Change Is Reversible transformation is not” – Ezgi Karaboya
I found the quote above deeply moving. It helped clarify where the guardrails around unwanted transformation should sit. It has also given me a kind of zen clarity about what I want Goblin to achieve: enable as much change as possible while establishing controls that prevent users from transforming anything in a way they cannot undo, especially without their knowledge.
I somewhat stumbled into the idea of writing Goblin code to file storage rather than using traditional storage media. It is one reason I write this blog and try to learn through practical experimentation: sometimes you discover ideas by accident, then realise you love them.
There is probably more work to do here. The read-file function could be improved by streaming file contents, stopping at a ‘;’ delimiter, and executing the code incrementally. That would limit how much memory a file can consume. However, this is already the bare beginning of the dedicated simulation language I wanted to build. I cannot think of a functional aspect of Python or SQL for which it does not have an equivalent approach—apart from multithreading, of course.
I want reversible code, as discussed above, to remain a core philosophy of the language. I know that storing databases in scripts may seem unusual, but scripts are readable and, in some ways, more self-documenting than SQL. Because Goblin is a graph database designed to reduce the need for joins, it may also make that work more accessible. I would like to retain script files and develop them as “transformation” files rather than simply executable files—for example: if this does not exist, create it.
I think more people should write a programming language. It is surprising how quickly you realise what you dislike about the tools you have used unquestioningly for years. I have been told that computer-security issues arise when you trust something you should not, or fail to control something you should. I want to embed that principle into Goblin: recognise that transformation can be irreversible, define clear boundaries around it, and create a safer space for coding.
Add comment
Comments