efene talk @ Erlang Factory Lite Brussels

I will be giving a talk on Erlang Factory Lite Brussels the 1st of February

the page:

http://www.erlang-factory.com/conference/Brussels

the talks:

http://www.erlang-factory.com/conference/Brussels/talks

the schedule:

http://www.erlang-factory.com/conference/Brussels/programme

see you there :)

macro functions in efene

yesterday I added support for macro functions both being defined in efene and being imported from erlang sources.

to get an idea on how it works, here is an example

emacro.erl

-module(emacro).
-export([run_erl/0]).

-define(PRINT(A, B, C), io:format("~p ~p ~p~n", [A, B, C])).

-define(BORING_ERL_CONSTANT, 42).


run_erl() ->
    ?PRINT(1, two, "asd").

fmacro.ifn

@import("emacro.erl")

$boring_fn_constant = 1337

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

$PrintReverseAndSayGoodBye = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])
    io.format("goodbye~n")

@public
run = fn ()
    $PRINT(1, asd, $BORING_ERL_CONSTANT)
    $PrintReverse(1, 2, $boring_fn_constant)
    $PrintReverseAndSayGoodBye("world", "macro", "hello")

compiling to erlang (to see the result)

mariano@ganesha:~/tmp$ fnc -t erl fmacro.ifn

-module(fmacro).

-export([run/0]).

-export([run_erl/0]).

run_erl() -> io:format("~p ~p ~p~n", [1, two, "asd"]).

run() ->
    io:format("~p ~p ~p~n", [1, asd, 42]),
    io:format("~p ~p ~p~n", [1337, 2, 1]),
    begin
      io:format("~p ~p ~p~n", ["hello", "macro", "world"]),
      io:format("goodbye~n")
    end.

compiling and running (to show that it works :P)

mariano@ganesha:~/tmp$ fnc fmacro.ifn
Compiling fmacro.ifn

mariano@ganesha:~/tmp$ fnc -r fmacro run
1 asd 42
1337 2 1
"hello" "macro" "world"
goodbye

mariano@ganesha:~/tmp$ fnc -r fmacro run_erl
1 two "asd"

nice error handling

here I first show a module with some error and then the error that appears when trying to compile it
 

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

@public
run = fn ()
    A = $PrintReverse

Compiling nicemacroerrors.ifn:

line 7: trying to use a macro function as a value?

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

@public
run = fn ()
    A = $PrintReverse(1, 2)

Compiling nicemacroerrors.ifn:

line 7: calling macro nicemacroerrors.'PrintReverse' that expects 3 args with 2 instead

$PrintReverse = fn (pattern_match=A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

@public
run = fn ()
    A = $PrintReverse(1, 2, 3)

Compiling nicemacroerrors.ifn:

line 2: all parameters in a macro must be variables

$PrintReverse = fn (A, B, C) when is_list(A)
    io.format("~p ~p ~p~n", [C, B, A])

@public
run = fn ()
    A = $PrintReverse(1, 2, 3)

Compiling nicemacroerrors.ifn:

line 2: macro can't contain guards

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])
    syntax error

@public
run = fn ()
    A = $PrintReverse(1, 2, 3)

Compiling nicemacroerrors.ifn:

line 4: syntax error before:  "error"

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

fn (A, B, C)
    io.format("another clause")

@public
run = fn ()
    A = $PrintReverse(1, 2, 3)

Compiling nicemacroerrors.ifn:

line 2: macro can't contain multiple clauses

$PrintReverse = fn (A, B, C)
    io.format("~p ~p ~p~n", [C, B, A])

$Constant = not_a_function

@public
run = fn ()
    $PrintReverse(1, 2, 3)
    $Constant(1)

Compiling nicemacroerrors.ifn:

line 10: trying to call macro nicemacroerrors.'Constant' but it's not a function

notes

erlang macro functions can be used as long as they are syntactically valid erlang (erlang doesn't enforce this since it handles macros at the lexical level AFAIK).

there is still some work to do to handle macro heavy modules like eunit, but it's looking good :).

more erlang integration

thanks to DavidMileSimon and vmx for providing feedback, ideas, issue reports and patches efene keeps moving near it's 1.0 milestone.

this time the improvements came in two fronts:

• rebar build
• @import, constants and erlang integration

on the rebar side now we can build the project following this steps:

git clone git://github.com/marianoguerra/efene.git
cd efene
./build.sh
./build.sh

you have to run build.sh two times since the first one fetches and compiles all dependencies and the compiler. The second run compiles efene libraries.

on the language side we now have access to constants defined in imported modules, both efene and erlang imported modules!

import also reports constant redefinitions providing the place where the constant was defined (even on imported modules, even on erlang modules :)

toimport.ifn

$foo = 4
$bar = 6

@public
to_import_ifn = fn ()
    io.format("i'm a function from module toimport.ifn~n")

asd.erl

-module(asd).
-export([runerl/0]).

-define(FOO, 4).
-define(BAR, "asd").

runerl() ->
    io:format("~p ~p~n", [?FOO, ?BAR]).

importall.fn

@import("toimport.ifn")
@import("asd.erl")

$baz = 1

@public
run = fn () {
    to_import_ifn()
    io.format("constans from this module ~p~n", [$baz])
    io.format("constans from toimport.ifn module ~p ~p~n", [$foo, $bar])
    io.format("constans from asd.erl module ~p ~p~n", [$FOO, $BAR])
}

running example

mariano@ganesha:~$ fnc importall.fn
Compiling importall.fn

mariano@ganesha:~$ fnc -r importall run

i'm a function from module toimport.ifn
constans from this module 1
constans from toimport.ifn module 4 6
constans from asd.erl module 4 "asd"

redefining an imported constant

importall.fn

@import("toimport.ifn")
@import("asd.erl")

$baz = 1
$FOO = "redefining"

@public
run = fn () {
    to_import_ifn()
    io.format("constans from this module ~p~n", [$baz])
    io.format("constans from toimport.ifn module ~p ~p~n", [$foo, $bar])
    io.format("constans from asd.erl module ~p ~p~n", [$FOO, $BAR])
}

the compiler provides details on constant redefinition

mariano@ganesha:~$ fnc importall.fn
Compiling importall.fn
line 5: constant 'FOO' already defined at {asd,4}

next steps

• migrate tests to a real testing library
• support macro functions
• add more tests and documentation for the new features
• improve error messages

Rebar support for efene - thanks to David Simon

This is a guest post from David Simon who implemented rebar support for efene, a round of applauses to him!

Rebar is a build and project-management system for Erlang, similar to tools such as SCons, autotools, and ant. It has a lot of cool features, including release packaging, templates, automatic dependency management, and support for various testing systems.

Recently, we've got Rebar to work together with Efene using a plugin, and also switched Efene from its custom build system to a Rebar-based one. This means that you can now use Rebar to manage your Efene projects. Specifically, you can now have Rebar:

• Automatically download and compile Efene for you.
• Compile Efene (.fn and .ifn) files just like any other Erlang source file.
• Depend on other libraries and apps which are written in Efene.

To do this, your project needs specify in its rebar.config that it uses rebar_efene_plugin and that it depends on Efene and the plugin:

{deps, [
 {efene, ".*",
   {git, "git://github.com/marianoguerra/efene.git", "master"}
 },
 {rebar_efene_plugin, ".*",
   {git, "git://github.com/DavidMikeSimon/rebar_efene_plugin.git",
"stable"}
 }
]}.

{rebar_plugins, [ rebar_efene_plugin ]}.

Once you've done that, you can run the appropriate rebar commands to download the dependencies and compile everything:

$ rebar get-deps
$ rebar compile

And you should be good to go! Any .fn or .ifn files in your src directory (or any other source directory you specify using the src_dirs directive in erl_opts) will be compiled as needed whenever you run rebar compile.

More detailed instructions on how to start an Efene project from scratch using Rebar are available in Efene's README file (viewable at https://github.com/marianoguerra/efene), under the heading "Using Efene in your app".

Unfortunately, we don't yet support EUnit tests written in Efene, as the EUnit module in Rebar is hard-coded to look only at regular .erl files. Look for this to be addressed fairly soon.

last call for efene 1.0, talk now!

I'm thinking on doing the 1.0 release of efene as the next release, for this I need to think on what's missing for a 1.0 release, my thoughts are:

• better documentation
• better error messages
• better support for tools (rebar)
• more tests
• mark some stuff as experimental (mainly meta-stuff)

as you can see until now it's a "conservative release", the last things that were implemented were a little "experimental", like polyglot imports and constants that will stay as is if no comments are done in that respect.

so I'm calling for interested people to get on board and help me define what will go into the first stable release of efene before it's too late.

because we know what happens when you have to support bad decisions in programming languages..

I'm listening :)

Rebar efene plugin

thanks to David Mike Simon we have a plugin for rebar:

https://github.com/DavidMikeSimon/rebar_efene_plugin

let's work on making efene a first class citizen on rebar :)

RFC: introducing (experimental) polyglot imports for efene

one of the remaining features to implement was include/import

this feature is another one that uses the pre processor in erlang, and since I don't want to implement a preprocessor I had to find a way to do it differently.

but still there was a thing...

the main use of -include is to include record definitions and those record definitions are written in erlang.

another obvious thing is that there is not much code on efene that declares records and efene provides structs, that don't require including external files, this make includes kind of useless in efene.

also there is this "erlang <3 efene, efene <3 erlang" that promotes complete interoperability between both languages, not being able to include erlang in efene would make less true...

last but not least, if I included an ifene module in an efene module "as is", this is, as text before compiling, it won't compile because they are different languajes. This would make include pretty useless.

to avoid all this problems, and avoid including a pre processor I implemented @import (efene version of -include) as an attribute that does some magic at compile time (like @type, @spec and @rec), basically what it does is:

1. guess the type of the file by it's extension
2. call the correct compiler depending on the type
3. get the AST of the imported module and clean up stuff like module and file definitions, @public attributes and eof marks
4. insert the remaining ast in the module where it was imported

here is an example

toimport.ifn

@public
to_import_ifn = fn ()
    io.format("i'm a function from module toimport.ifn~n")

toimport.fn

@public
to_import_fn = fn () {
    io.format("i'm a function from module toimport.fn~n")

}

toimport.erl

-module(toimport).
-export([to_import_erl/0]).

to_import_erl() ->

    io:format("i'm a function from module toimport.erl~n").

toimport.hrl

-module(toimport).
-export([to_import_hrl/0]).

to_import_hrl() ->

    io:format("i'm a function from module toimport.hrl~n").

importall.ifn

@import("toimport.ifn")
@import("toimport.fn")
@import("toimport.erl")

@import("toimport.hrl")

@public
run = fn ()

    to_import_ifn()
    to_import_fn()
    to_import_erl()
    to_import_hrl()

here we declare 4 modules to be imported by a module called importall, each module is written in a different language (well, hrl is erlang..)

we import all and use the functions, nice uh?

now compile and run

$ fnc importall.ifn 
Compiling importall.ifn

$ fnc -r importall run
i'm a function from module toimport.ifn
i'm a function from module toimport.fn
i'm a function from module toimport.erl
i'm a function from module toimport.hrl

other cool things that this allows is to build your own importers on top of the default imported, do import and transformation/filtering on the imported code and any thing you can think of

we could easily do an import version that allows to load modules from the web, from a database or some other place..

even imports that allow to load DSLs and compile them to erlang bytecode, the sky is the limit!

let me know what you think