“Hard drives never go bad, They just become write only” – IT humour
Introduction
I am building my own programming language, with the goal of combining the flexibility of a scripting language with the power of a database query language. While it is primarily a learning project, I hope to use it for other tasks in the future.
The working name for the language is goblin.
My current focus is on disk storage: saving and restoring data files. The simplest approach is to use write-ahead files, which record every command a user makes and write those commands to disk. There will be edge cases and additional commands to handle, but if every database change is captured, these files can be used to undo many mistakes, apply updates, create backups, and restore the database to its exact state in most situations.
Keeping a complete log indefinitely may not be desirable, so the language will also need commands that can rewrite these files into a more compact form.
It can now open in a terminal, with input displayed in white and output in cyan. It also follows the REPL (read-eval-print loop) model, meaning it can be used from most terminals. The eventual goal is to open a command prompt, type goblin, and have an all-in-one database, scripting, and analytics sandbox available until you type quit. There is still a long way to go, but the white-and-cyan interface is already working well, and the remaining colours are reserved for different database responses.
Errors will definitely be displayed in red.
A visual guide is here : I Wrote A Programming Language - Starting Goblin
Philosophy
I want to create a safe scripting and database language that I could give to my three-year-old child, confident that he would not be able to do anything “bad” with it.
To achieve this, I want to remove anything that can interpret external code. In practice, I have never needed that capability; it seems like an overly elaborate way to create functions.
The current design is complete, but it needs substantial testing before next week. It is based on the idea that the entire codebase can reverse-engineer itself into native Goblin code and store that representation on disk. The process handling still needs some work, but the intended workflow is straightforward: when the program opens, it reads a run file and loads the full database; it then records every change in a write-forward file; and, on a formal quit, it consolidates the database into a complete saved state.
There are still several decisions to make about the best way to handle this process and its testing. At present, however, each database component knows how to write the code needed to recreate itself.
This approach is functional, though primitive. Since the language is intended to sit somewhere between a database query language and a scripting language, it should be possible to write a consolidated script to disk that fully recreates the current state.
Although this may sound primitive, the underlying principle is widely used. PostgreSQL enforces write-ahead logging, and Apache Kafka uses logs that can be distributed across multiple disks. This approach should provide reliable basic disk persistence while following a proven, ubiquitous pattern.
Run File On Start, Run File On Exit
When I open Python, I usually need to run a series of imports before I can begin. These initial commands load the required tools, configure the environment, and prepare everything for use.
In Goblin, I would like to automate this process. When a database is opened, Goblin should be able to run a start up file that handles imports, data loading, and other initial setup tasks. The simplest approach would be a file written in Goblin syntax: a customisable setup file for each database containing the commands needed to load tools, apply preferences, and retrieve data automatically.
Likewise, a file that runs when the user types quit could manage saving, clean-up, and other shutdown tasks. This file could also be written in Goblin syntax.
If every command is recorded as well, users should be able to reconstruct their work from these three files even if a session crashes—for example, because of a power loss.
This simple 1 file for start, 1 file to define close, and 1 file to bridge the gap in between sounds simple but there is a lot of interplay that can happen there. A setup file can also limit you and a file that does clean up work can be used to do "work". A table of data can be shipped off in a close down file or put through a pipeline of processes to do final checks.
Because there are only three editable files, users could maintain complex setup code in a straightforward way. Frequently used code, tools, and data-loading commands could be added to the start up file, so that each time a database is opened, it is already configured for that user. Since these scripts are written in Goblin, users would only need to learn one language.
For enterprise use, these files could be stored in administrator-controlled locations or protected through locking and permission controls. The exact implementation would need to be considered alongside security and cyber-risk requirements. This could allow Goblin to apply appropriate restrictions when an employee opens a database, limiting access to only the files, data, and actions they are authorised to use.
When a database is closed, the shutdown file could automate saving and related tasks. Because it is also Goblin code, it could be deeply customised—for example, to back up files and send them to the appropriate locations.
If Goblin is treated as a database system operated through terminals and command lines, automatically running clean-up tasks when a user disconnects would make it possible to automate important operational steps. With groups and rules that run when data is added or removed, closing scripts could also include classification logic and ensure that data is saved or routed to the correct locations.
Goblin also needs a way to capture the current setup and write it out as Goblin code. Because Goblin stores entry and exit queries for functions and groups, it should be possible to reverse-engineer that configuration into Goblin syntax and save it to a file.
Ideas For New Keywords
write saves data and creates a file. When followed by only a filename, it writes all available data to that location.
write [global] filename writes a specific global value—such as a function or group—to a specified location. The order of operations used to create this data may need further consideration.
run 'filename' loads the code from a file and executes it, creating the intended behaviour.
read 'filename' prints the contents of a script to the terminal, allowing you to review the code before running it.
cannot can be added to the startup document to disable specific language features. For example, a setup script could include a restriction such as cannot publish, allowing the terminal to run approved scripts while preventing it from generating new code. If this file is also read-only, users cannot modify these restrictions, giving schools or IT departments multiple ways to control the environment.
directory 'filepath' sets the file location. Combined with cannot, it can help create a secure sandbox environment. If no directory is set, the language defaults to the current system-level directory.
Conclusion and self criticism
I believe this lays the foundation for a fully personal database. That said, it is important to recognise that everything is stored in plain-text files, so it would not be appropriate for sensitive or personal data. However, given Goblin’s original brief as a personal database and scripting environment, this represents a significant step forward.
Alarmingly, my research found that most databases are not encrypted at the disk level, and only a small number of vendors enforce encryption during installation. Because encryption can slow down read and write operations, organisations often try other security measures first. There is also a frustrating challenge: many databases need encryption keys readily available, which can result in keys being stored within the database itself—valuable information for an attacker.
As a result, encryption is less common than you might expect. The bare-bones approach I described earlier is therefore not necessarily wrong at an enterprise level, although many organisations may be reluctant to admit it.
name=='alice' [head_o_tree]=(pop 0) popout 0 for ([helper] = alias [alice_temp] = () with[head_o_tree] while (? .right_path ? .left_path)( if ([helper] > alias)(~right_path)(~left_path))
if (? .right_path)(if ([helper] > alias)([temp] = .left_path = .right_path = [alice_temp] = [temp])(.left_path = [alice_temp]))(.right_path = [alice_temp])))
Appendix
Rather than returning data in the way Python does, my language is designed around building pipelines. It provides (), a calculation that works with the existing context, and {}, which creates a closed context.
I have also added subcalc, written as {), which overrides and rewrites the current context, and calcquery, written as (}, which accepts the current context but does not modify its values when control is returned.
This creates some interesting behaviour for loops. When a for loop uses a calculation—that is, ()—any operation that changes the context or pulls in additional data adds that data to the current context.
Calcquery, written as (}, simply receives each node in the dataset and performs the required code without changing the current context.
Subcalc, written as {), uses a small workaround to support this behaviour. It takes the input from the for loop, runs the code against it, and returns the results along with a new context. However, that context contains only the values that changed. For example, if you use a calculation to look up relationships, the newly discovered results are added to your data. Use calcquery when you want to ensure the current context remains unchanged, and use subcalc when you want to replace the current context with only the new changes.
This is particularly useful when creating nodes and relationships. () returns the newly created node, whereas (} preserves the current context. For example, Connect () connect () connect () links nodes into a one-way list. ( Connect () connect () connect () } creates the same list, then returns to the original context. ( Connect (new) connect (new) connect (new) connect (new} connect (new}} creates a list, terminates it with two new edges, and then returns the context to its original starting point.
Value assignment follows the behaviour you would expect based on the leading value. For example, [global]=() and [global]=(} take the current context and update the global value. In contrast, [global]={} and [global]={) ignore the current context and run afresh.
While has changed significantly. () now takes the current context and reruns its while loop using the output of the repeated while code. (} takes the initial context, but repeats from there.
Now
while()() the data flows back and forth between both cells. Processing can change context in either.
While(}(} the data takes the original context and never changes it.
While{}{} all context is decided between the {}
While{)(} first cells choses the nodes without looking at outside data and passes it to the second for changes.
While{}{) the first decides if to continue but does not listen to anything outside it. Hen the code finishes the last context is what the data will take.
While{){) the first decides if to continue and either cell can change context. If()(){) could be interesting because can look at the logic and if something does not exist refresh to a new context. You can see it gets very flexible.
The same applies to delete operations and similar actions. If you can use a calculation or subquery without needing to change the context, it does not require any special handling.
I initially wanted to reverse the symbols, using { for a permissive wall and ( for a rigid wall. To me, { resembles a small gate or an enzyme entry point in a living cell. However, mathematics already uses ( for more permissive or loosely defined groups, while [ and { tend to indicate groups that are more rigidly defined or fixed. So I have stayed with what feels like the more mathematical notation, even though mathematics does not use mixed (} or {) groups.
More broadly—and this may be the part that makes senior developers, who may have dismissed this section so far, take notice—this notation offers a simple way to investigate unexpected behaviour. If the code does something surprising that you do not understand, work through each ) sequentially and change it to a }. The point at which changing ) to } changes the output identifies the cell responsible for the fault. If none of the changes affect the result, the issue lies outside any cell or may be a data problem.
I also like how well this relates back to mathematics. When looking at code, it suggests that the ( and ) boundaries are the areas most likely to alter control-flow behaviour.
I am not addressing cross-query folding with this concept, although it could potentially be extended to cover it. For now, I think cross-query folding should remain separate; combining these ideas could become confusing. Expressions such as {}+{} and {}/{} imply two tightly bounded groups and an operation performed across them. By comparison, ()+() combines two fuzzy groups, while ()+{} combines a fuzzy group with a rigidly defined one. That distinction feels important, although I am not yet certain why.
Functions are the last area I am considering. A function should simply be a reusable piece of code. Its definition would look like this: [global]:()(). The calculations within () should, logically, remain permeable in line with the symbolic logic adopted so far.
I could make functions behave in the same way, but I am concerned that changing a function—or forgetting what it does—could unexpectedly modify the surrounding context. Treating functions as context-neutral by default seems safer. Nothing would prevent wrapping a function in (} in the future, allowing users to inherit context from a function by choice rather than by the author’s design. I lean towards this approach because, when you are uncertain about code, you should be able to run it and see what comes out the other end. Ideally, this ensures that nothing becomes a black box. For now, a function simply stores the code text.
join allows a set of data to join a group. This has now been modified so that you can join through a calculation or subcalc—() or {). Once the new data has joined and the group’s setup rules have been applied, the context shifts to the group’s data. This means you can then test how the group-classification rules behave.
If you want to force any of these constructs to run, use with: with () runs a calculation; with {} runs a subquery; with (} runs a calcquery; and with {) runs a subcalc. I decided this was the best approach because the language relies on dyads and triads to make intent explicit; running these constructs on their own would be an outlier in the language.
There may be one exception: calling a function and listing its arguments afterwards. However, that is user-defined rather than part of the core language. Semantically, with works particularly well with calculations () and subcalcs {), because you are genuinely adopting a new context and doing something “with” it. It is a slightly less natural fit for subqueries {} and calcqueries (}, where you do not necessarily act “with” the output, but I have chosen to overlook that minor inconsistency.
If I simply want to run code without changing the current context, ([function_name]} takes your data, processes it, and then returns you to what you were doing. {[function_name]) lets you use a function to retrieve data while changing context. ([function_name]) is equivalent to [function_name], while {function_name} isolates and runs code independently, ensuring that nothing outside it can enter or leave.
At a high level, ( and ) have become permeable boundaries, whereas { and } are not. You can combine them in many ways: ( creates a function that accepts outside data, ) replaces a return value, { indicates that no outside input is accepted, and } indicates that nothing is returned. Combined, they can produce some interesting outcomes.
This project has become more important to me since my PC crashed. After updating to Python 15, I encountered a deallocator issue with the SQLite3 dependency. It reinforced an old-man belief I have developed about code: a quiet aversion to relying on anyone’s code or infrastructure that I could not build or reverse-engineer within a week or two.
PS: I’m not entirely sold on the names calcQUERY and SUBcalc either. I designed the language around permeable calculations and impermeable subqueries to enforce encapsulation, and the names simply stuck. That’s the beauty of writing your own language: even if you’re objectively wrong, there’s no one around to tell you otherwise.
Add comment
Comments