metalanguage and language set

Extracting a language from java helloworld

This tutorial demonstrates the language extraction method first presented at OOPSLA 2007. We'll extract a language from a java helloworld application. The language, hello, can be used to generate more greeting applications. We'll first write an example of the kind of application we want to generate. Then we'll convert it to ngr and extract the new language from it by parameterization.

Write the example application

~/work $ mkdir extracting-hello && cd extracting-hello
~/work/extracting-hello $ mkdir -p src/hello
~/work/extracting-hello $ $EDITOR src/hello/Hello.java

package hello;

public class Hello {

  public static void main(String[] args) {
    System.out.println("Hello World!");
  }

}

Convert the application to ngr

~/work/extracting-hello $ java2ngr --srcdir src > hello-java-sourcemodule.ngr

Verify java generation

~/work/extracting-hello $ $EDITOR hello-java-sourcemodule-shar.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/net/sf/ngrease/languages/java/context.ngr
    $:evaluate:$:reference:resource:/net/sf/ngrease/languages/shar/context.ngr
  }
):with {
  parent-dir:"."
  $:transform {to {java-source-directory directory sh-source} from:
    $:reference:resource:/hello-java-sourcemodule.ngr
  }
}

~/work/extracting-hello $ export NGREASEPATH=.
~/work/extracting-hello $ ngrease -f hello-java-sourcemodule-shar.ngr > hello-java-sourcemodule.shar

(Output asserted)

~/work/extracting-hello $ mkdir tmp && cd tmp
~/work/extracting-hello/tmp $ bash ../hello-java-sourcemodule.shar
~/work/extracting-hello/tmp $ javac src/hello/Hello.java
~/work/extracting-hello/tmp $ java -classpath src hello.Hello
Hello World!

(Output asserted)

~/work/extracting-hello/tmp $ cd ..

Test the hello language stub

~/work/extracting-hello $ $EDITOR hello-language-test.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/hello-language.ngr
  }
):and {

  #:"The example case"

  $:assert:$:equals {
    $:reference:resource:/hello-java-sourcemodule.ngr
    $:transform {to:java from:
      hello
    }
  }
}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Cannot load reference /hello-language.ngr

(Output asserted)

The test failed because of the missing hello language context so we'll create an empty context.

~/work/extracting-hello $ $EDITOR hello-language.ngr

# ngrease 1

context {

transformers {

}

}

We run the test again to see we still don't have a transformer from hello to java.

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Don't know how to transform to java from hello

(Output asserted)

Make the test pass with a constant transformer

We copy-paste (formatting whitespace a bit) the example to a constant transformer.

~/work/extracting-hello $ $EDITOR hello-language.ngr

# ngrease 1

context {

transformers {

hello {
  java:
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello World!'
              }
          }
        }
      }
    }

} #:hello

}

}

This time the test passes, and we now have a working constant language.

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Don't know how to output result: true

(Output asserted)

Parameterize the message to print

A constant transformer makes a quite boring language so we'll write a test for passing the message to print as a parameter.

~/work/extracting-hello $ $EDITOR hello-language-test.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/hello-language.ngr
  }
):and {

  #:"The example case"

  $:assert:$:equals {
    $:reference:resource:/hello-java-sourcemodule.ngr
    $:transform {to:java from:
      hello
    }
  }

  #:"Different message"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello Dagobah!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {message:'Hello Dagobah!'}
    }
  }


}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr 2>&1 | head -n 2
Assertion failed for: $:equals {

(Output asserted)

We make the test pass by replacing the hard-coded message with a value read from the source element. Now that the transformer body is no more a constant, we need to quote it.

~/work/extracting-hello $ $EDITOR hello-language.ngr

# ngrease 1

context {

transformers {

hello {
  java:
  java:$:quote:
  $:with {
    $:child {symbol:message of:$:source}
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello World!'
                string-constant:$:message
              }
          }
        }
      }
    }
  }

} #:hello

}

}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Cannot find child by symbol message from hello

(Output asserted)

Now that our language requires the message parameter we need to update the first test.

~/work/extracting-hello $ $EDITOR hello-language-test.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/hello-language.ngr
  }
):and {

  #:"The example case"

  $:assert:$:equals {
    $:reference:resource:/hello-java-sourcemodule.ngr
    $:transform {to:java from:
      hello
      hello {message:'Hello World!'}
    }
  }

  #:"Different message"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello Dagobah!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {message:'Hello Dagobah!'}
    }
  }


}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Don't know how to output result: true

(Output asserted)

Parameterize the stream to print to

Now we test stream parameterization. This time we leave the other tests as such on purpose, since we want to make this new parameter optional. Most users want their greeting printed to stdout anyway.

~/work/extracting-hello $ $EDITOR hello-language-test.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/hello-language.ngr
  }
):and {

  #:"The example case"

  $:assert:$:equals {
    $:reference:resource:/hello-java-sourcemodule.ngr
    $:transform {to:java from:
      hello {message:'Hello World!'}
    }
  }

  #:"Different message"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello Dagobah!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {message:'Hello Dagobah!'}
    }
  }

  #:"Different stream"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System err}
                println
                string-constant:'Hello stderr!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {
        message:'Hello stderr!'
        stream:field-ref {variable-ref:System err}
      }
    }
  }

}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr 2>&1 | head -n 2
Assertion failed for: $:equals {

(Output asserted)

We implement the feature by checking for the existence of the parameter and using System.out as default.

~/work/extracting-hello $ $EDITOR hello-language.ngr

# ngrease 1

context {

transformers {

hello {
  java:$:quote:
  $:with {
    $:child {symbol:message of:$:source}
    stream:
      $(@):if {
        $:child-exists {symbol:stream of:$:source}
        $:child {symbol:stream of:$:source}
        stream:field-ref {variable-ref:System out}
      }
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                $:stream
                println
                string-constant:$:message
              }
          }
        }
      }
    }
  }

} #:hello

}

}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Don't know how to output result: true

(Output asserted)

Raise abstraction of the stream parameter

The stream parameter is still a java code fragment. We would like to make the language more abstract, i.e. use only concepts from the helloworld domain, so we'll modify the test to use the general terms stdout and stderr instead of their java implementations.

~/work/extracting-hello $ $EDITOR hello-language-test.ngr

# ngrease 1

$ (
  context-chain {
    $:default-context
    $:evaluate:$:reference:resource:/hello-language.ngr
  }
):and {

  #:"The example case"

  $:assert:$:equals {
    $:reference:resource:/hello-java-sourcemodule.ngr
    $:transform {to:java from:
      hello {message:'Hello World!'}
    }
  }

  #:"Different message"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello Dagobah!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {message:'Hello Dagobah!'}
    }
  }

  #:"Different stream"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System err}
                println
                string-constant:'Hello stderr!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {
        message:'Hello stderr!'
        stream:field-ref {variable-ref:System err}
        stream:stderr
      }
    }
  }

  #:"Explicit stdout stream"

  $:assert:$:equals {
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                field-ref {variable-ref:System out}
                println
                string-constant:'Hello stdout!'
              }
          }
        }
      }
    }
    $:transform {to:java from:
      hello {
        message:'Hello stdout!'
        stream:stdout
      }
    }
  }

}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr 2>&1 | head -n 2
Assertion failed for: $:equals {

(Output asserted)

We make the test pass by using and defining java transformers for the stream concepts.

~/work/extracting-hello $ $EDITOR hello-language.ngr

# ngrease 1

context {

transformers {

hello {
  java:$:quote:
  $:with {
    $:child {symbol:message of:$:source}
    stream:
      $(@):if {
        $:child-exists {symbol:stream of:$:source}
        $:child {symbol:stream of:$:source}
        stream:field-ref {variable-ref:System out}
        stream:stdout
      }
    java-source-module {name:src
      package {name:hello
        class {public name:Hello
          method {public static returns:void name:main
            parameter {type:String[] name:args}
            body:
              method-call {
                $:stream
                $:transform {to:java from:$:stream}
                println
                string-constant:$:message
              }
          }
        }
      }
    }
  }

} #:hello

stderr:java:field-ref {variable-ref:System err}

stdout:java:field-ref {variable-ref:System out}

}

}

~/work/extracting-hello $ ngrease -f hello-language-test.ngr
Don't know how to output result: true

(Output asserted)

Now the hello language has abstracted pretty much all there is to a helloworld application and we can start generating countless applications and make money...

~/work/extracting-hello $ cd ..