Creating a scripting language

As part of a revitalisation project for a complex system at work, I’ve concluded that one of our sustainability requirements could be best served through use of a specialised, extensible, implementation-independent scripting language.

Since programming language theory is something that’s always interested me, I wanted to make what I’m learning accessible to anyone else who might find it beneficial. Chances are this mostly includes computer science students encountering recursive-descent parsing for the first time or those in an entry-level compiler-construction class, so I’ll be writing towards that knowledge-set, but anyone sufficiently curious should be able to follow along.

The scope of what I’ll be publishing won’t be exhaustive (since it’s for work, I’ll need to keep our lexicon, which covers the function libraries that actually do stuff, private), but I will go as far as providing both a complete grammar and a functional interpreter that can perform some primitive operations, like arithmetic. The idea is that whatever I provide should be usable as a basis for another mostly open project (in any field — this language is geared towards blocking I/O operations, but it’s a very open shell), to help encourage the development of even more freely available, highly extensible systems.

Like everything published on this site, the language-spec and supporting materials are under the Creative Commons Attribution-Share Alike 3.0 License. This should pose no problem to academics looking to extend or learn from the language, so long as sources are properly cited. Any commercial parties interested in using this language for their own purposes will be free to do so, as long as they keep their product separate from the provided implementation — when the interpreter is published, it will have a standard API that allows arbitrary external functions to be invoked, enforcing abstraction of language from environment; as long as this relationship isn’t violated, the CC-3.0-BY-SA-licensed stuff can exist alongside proprietary, closed code, where the business logic resides, with the caveat that any distribution will require the language spec, and any modifications, to be included, in full source form.

The concepts that will be presented are portable to any LALR parsing engine, and very easily so to anything that implements BNF for its grammar rules, so moving from Python to another language should be a relatively easy, if tedious, task. Any such derivations will still be subject to the licensing terms (the idea is what the CC license protects, moreso than the implementation), so it’s strongly advised that you retain the gateway-API model, to avoid any unexpected licensing concerns if you have a need to keep your logic closed. The only foreseeable challenge with moving from a dynamic language to a static one (like Java or C#) is that you will need to have an external processor for scoped function-calls, since everything uses keyword arguments for forwards- and backwards-compatibility reasons: you’ll need to marshall attribute targets to fixed positional targets in your own namespace.

If you’re cool with the licensing, you can grab the code from its home on Launchpad, where the public interface will continuously be developed; feel free to fork it at any point. If you’re not planning to use it, but rather just want to learn the concepts, that’s good, too. (Just be sure to cite this series in your references)

Leave a Reply

Your email address will not be published. Required fields are marked *