Ogma v0.3 release - now with type inference

Version 0.3 of ogma introduces a pivotal new feature: Type Inference! With 15,000 insertions and just short of 10,000 deletions, this version was a major technical achievement with a complete rebuild of the compilation engine. This release also addressed some accumulated technical debt, restructuring the crate such that the compilation engine acts as its own module, and splitting the instrinc command implementations into command categories. These changes should assist future development as ogma grows.

Typing

---

ogma is strongly typed. This means that each node in the syntax tree has a known type at compile time, and the correct type must be used for inputs and arguments. ogma will try to guess the types being used in an expression using type inference, which, as the name implies, will infer the types from the usage. It is easy to show the type system throwing an error:

>> \ 2 | + 'a'
Semantics Error: expecting argument with output type `Number`, found `String`
--> shell:9
 | \ 2 | + 'a'
 |          ^ this argument returns type `String`
--> help: commands may require specific argument types, use `--help` to view requirements

You cannot add a string to a number!

The type system also works to choose the correct operation. For example, the len command will work on a table or a string:

>> \"Hello, world!" | len
13

>> Table | len
0

>> \3 | len
Semantics Error: `len` does not support `Number` input data
--> shell:5
 | \3 | len
 |      ^^^
--> help: use `len --help` to view requirements. consider implementing `def len`

In pre-0.3 versions, commands that supported varying types required that the types be explicitly specified. This was due to the implementation of the compiler, which worked linearly through each block, so only had type knowledge of previous blocks. With 0.3, ogma can deduce the types from the usage by considering each block as a distinct compilation unit and contextualising outwards.

# pre-0.3 -- type would have to be specified
ls | filter type --Str = file
                 ^^^^^ type flags were required

# with 0.3 -- types can be inferred
ls | filter type = file
                 ^ the input is inferred to be Str

Type Inference

---

To guess the types from use, ogma must trial differing types and use deduction to land on a type set which would fit an expression. This process is fallible, especially when the use would lead to ambiguity:

>> Table | nth 0 get foo
Semantics Error: ambiguous inference. more than one output type can compile op
--> shell:14
 | Table | nth 0 get foo
 |               ^^^ this op can be compiled with `Nil` and `TableRow` as output types
--> shell:14
 | Table | nth 0 get foo
 |               ^^^ try annotating output type: `get:<type> ...`

In this example, the get call has to infer an output type, but this could be any type! The inference algorithm is backed by graphs, and is quite robust, but not omniscient!

To help with type debugging, there is the typify command, which will annotate an expression with the resolved type graph.

>> typify { Table | nth 0 get:Num foo }
{:Nil Table |:Table nth 0 {:TableRow get foo }:Num }:Num

Type Annotation

---

v0.3 introduces syntax to annotate type information, which can help the type inferencer resolve. Annotations act as constraints on the typing. The syntax consists of a leading colon and then the type name :<type>. An annotation before a command constrains the input, while an annotation after a command constrains the output. Expression outputs can also be annotated.

>> {:Num Table |:Table nth get:Num foo }:Num
    ^^^^        ^^^^^^        ^^^^      ^^^^
    |           |             |         | constrain expression output
    |        |          | constrain command output
    |        | constrain command input (`nth command`)
    | constrain command input

Annotations can accidentally be applied which results in type conflicts:

>> Table |:Num + 3
Typing Error: Type resolution failed. Conflicting obligation type
--> shell:0
 | Table |:Num + 3
 | ^^^^^ this node returns a `Table`
--> shell:12
 | Table |:Num + 3
 |             ^ but this node is obliged to return `Number`

Looking Forward

---

Landing such a major change will result in some unforeseen bugs arise, so the focus over the coming months will be to iron out any kinks with the new feature. There has also been a backlog of new commands and features which have been held back, waiting for this feature to land, which can now continue with development. Finally, namespaces, the next major feature, can begin the prototyping phase.

ogma is open source. If you like ogma, please support its development through code contributions or sponsorship.