DEVELOPER'S CORNER

Status

Speagram is currently in alpha stage and is not ready for production use. Still, most of the features already work quite stable and we are starting to plan a beta release. Speagram developement started in 2003 and since the first release in June 2005 we are trying to make a release at least twice a year. Release history:

• Speagram 0.1 (alpha)  —  June 14, 2005
• Speagram 0.2 (alpha)  —  December 22, 2005
• Speagram 0.3 (alpha)  —  April 28, 2006

Speagram source code is available under the BSD licence from

• Speagram SourceForge repository
• our download page.

Major Tasks and Problems

• We need to improve our web interface and application integration. The web interface still has a few bugs and does not warrant appropriate security and scalability. Online files need improvement to be used for editing, for example each part should be processed by Speagram separately and asynchronous server calls can be used in a few places. It should also be possible to access Speagram as a web service and a UDDI file should be present. Moreover, Speagram should be able to communicate with other web services and execute them during normal Speagram function execution. This is not a trivial problem as it must be correlated with internal memoization and multithreading features. Moreover, we would need XSLT support in the basic library to make it practical to use web services from within Speagram.
• Our internal rewriting engine needs improvement. Currently it is fully interpreted which has performance disadvantages. We want to use generative programming (most likely MetaOCaml) to overcome this problem and we thought about it a lot during our design, but did not implement it yet. Moreover, currently we handle only flat functional values and it would be nice to have automatic closures. Our heuristics for automatic memoization and multithreading work quite well but should be re-thought if we want to handle large grammars with very complex disambiguation and transformations.
• The way our parser reports errors must be improved. Internally the parser gathers very precise information about what parts of the sentence were parsed, what are the possible results and how could these possibly be prolonged to a complete parse. Unluckily most of this information is not used because to present it in a meaningful way we need to add sensible post-processing. This has to be implemented. Moreover, the parser should be able to correct simple errors in the input and accept the parse even if some parts are missing or garbled, for example misspelled.
• Our implementation of logic tools is unstable. We are getting good results with the algorithms we use and find our design quite capable, but the implementation is immature and was extensively used for testing and never focused on stable use. Most probably the best way to go is to completely rewrite the whole part that concerns logic and this time ensure that polymorphic types and functional values are handled correctly. This work has started but it is in very early stages.
• Speagram is not a probabilistic program in any way, meaning that we are not using word counts or keywords in any way and only look for the well-defined right parses and results. While we want to stick to this core idea, we think that there is a number of very interesting ways to use statistical methods inside Speagram. The first way that we would like to try is to implement a kind of automatic disambiguation based on how frequently the alternatives occur on the net or in some data bases. There are also other more complex and very interesting ways to use probability in Speagram that we would like to investigate, both for the parser and for our logic and search tools.

Installation Guide for Developers

Normally Speagram developement takes place under Linux. You can make analogous steps under Windows but our Makefiles rely on command-line tools like sed, diff, cp, rm and on bigger packages like ocaml and latex. Installing them on Windows and adjusting the Makefiles might be troublesome. The best method is to install Cygwin — everything is going to work then just as it works under Linux.

• Ocaml
  Install complete Objective Caml version at least 3.07 from www.ocaml.org. The installation must include OCamlP4 (it normally does).
• Subversion
  First you must have an account at sourceforge.net. Suppose that your sourceforge.net login is xxxcoder. Checkout the speagram svn repository (only trunk/speagram necessary):

  svn checkout https://svn.sourceforge.net/svnroot/speagram/trunk/speagram

  Then work inside your_dir/speagram/. To check-in modified files do

    svn up     // to update to the latest version from repository
    svn stat   // to see what files have changed, which are new etc.
    svn commit --username xxcoder -m "Write short changes description here"

  After your first commit you will not be asked for username and password any more, so you can just do svn commit -m "Message". We are still using .cvsignore files as we find them easier to manage than svn properties. Therefore if you change .cvsignore do the following:

    svn propset svn:ignore -F .cvsignore

  For more help on using SVN with sourceforge you can read the following.
  • Linux or Command Line: http://sourceforge.net/docs/F06/
  • Windows: http://sourceforge.net/docs/F07/
• Configuration file
  Look at the file config.mak in /home/your_user_name/speagram/. It does not need to be changed under most Linux systems, but it might happen that you will have to adjust it. If you have tweaked your Linux to this point you probably know how to adjust this file.
• Compiling Speagram
  Go to speagram/src and then type the following commands

    make             // compile Speagram from source
    make parsed      // build the Speagram library
    make check       // run tests to check if everything works

  Now you are ready to run speagram in src directory.
• Building documentation
  To build speagram documentation you need to have a decent LaTeX 2e installation, which is normally present on most Linux systems. To build the dvi, ps and pdf files with the documentation you should go to /home/your_user_name/speagram/doc/ and type make.
• Creating distribution tarball
  When you have compiled speagram and built the documentation you can go to /home/your_user_name/speagram/ and type make tarball. This will create the full speagram distribution tarball.