Creating an ATOMS toolbox

This page describes the usual method to create, package, and distribute a Scilab external module (ATOMS Toolbox).

In the first part, we show how to structure the module by creating directories, sub-directories and files at the right place. Then we customize the templates to create the builder(s), the loader(s), and the Scilab help files. In the last section, we present how to upload the module on the Scilab website.

The examples in this document are provided in the module skeleton provided with Scilab. This module is a template of module that is located in SCI/contrib/toolbox_skeleton (or on the website). To get this example on Windows, you must install "A toolbox skeleton (to extend Scilab)" component when we install Scilab.

You can see also these other pages:

Contents

General concepts
1. Concepts
2. Toolbox structure
Build and load a module
Uploading your toolbox
Q&A

General concepts

Toolboxes have their own terminology for a few concepts; here are the words that you will use used in the documentation, and the convention for the toolbox structure and files.

Concepts

Script

A Scilab script is a text file having the extension .sce (see example), used to store Scilab code, and which can be executed using the exec function.

Macro

A Scilab macro is a Scilab function written in Scilab code and stored in a file having the extension .sci (see macro).

Primitive

A Scilab primitive is a Scilab function whose implementation is done in a native function, in C, C++, even Fortran, or any other language compiled natively. A native function is called through a Scilab gateway.

Gateway

A Scilab gateway is a function, written in C or C++, that wraps a native function so that it can be called from Scilab as a primitive (by extension, the term gateway is sometimes used for all gateways provided in a toolbox). The native function can either be a symbol defined in an external library, or a function whose source is provided and compiled into a library at the beginning of the toolbox's compilation. A gateway is responsible for converting the data transmitted to and from the native function, and calling that native function. Once a gateway is declared in the builder script, built and loaded, the primitives are accessible directly from Scilab.

An example of a gateway can be found in the skeleton for new toolboxes: C gateway.
- It uses the Scilab C API for input and output parameter access and manipulation, and for error reporting.
Another example of a gateway written in C++ is also provided in the toolbox skeleton: C++ gateway.
- It makes direct calls to Scilab in C++.

See also: How to create a gateway.

Builder

A builder file is a Scilab script used to build the module i.e. to create a binary version which Scilab can load from a source version. This file is named builder*.sce (Examples: builder.sce, buildhelp.sce, ...).

Loader

A loader is a Scilab script which loads a toolbox component (or the whole toolbox) in Scilab. The loader names have loader*.sce (Examples: loader.sce, loadhelp.sce, ...). This file is usually generated by the corresponding builder file; as a toolbox developer, you don't need to create one.

Toolbox start & quit

Every installed toolbox is started at Scilab startup, and quit when Scilab is quit. When a toolbox is started (respectively quit), a toolbox initialization (respectively finalization) script is executed.

Toolbox structure

By convention, toolboxes have the following structure. They could have others, but all toolboxes hosted on atoms.scilab.org MUST follow it.

The toolbox skeleton, can give you an idea of the structure of a toolbox; all files are not needed, though.

By convention, the root directory has the generic name of the module (here toolbox_skeleton). Its sub-directories should be:

Directory	Content	Required or optional ?
macros	macros (.sci), macros builder (buildmacros.sce)	required (your toolbox would be useless without any macros...)
src	source code files (.c, .cpp, .f,...), and source builder & loaders split (loader.sce), in several sub-directories for each language (c, cpp, fortran*)	only if the toolbox has such source code
sci_gateway	source code files of gateways (.c, ...) and gateway builder & loader (.sce)	only if the toolbox has src source code
jar	java packages, and help files (.jar)	not in the toolbox source: generated by the build
help	XML help files (.xml), split in several sub-directories for each language (en_US, fr_FR,...)	strongly suggested
etc	initialization (.start) and finalization (.quit) scripts	modulename.start is mandatory, noticeably to run tbx_make()
tests	module test scripts (.tst)	strongly suggested
demos	some script examples to illustrate your module (.sce)	optional
includes	header files that you want to publish with your module (.h)	optional

and it should have the following files:

File	Description	Required or optional ?
readme.txt	installation guide, etc.	optional
DESCRIPTION	description of the toolbox; see the sample in toolbox skeleton for its syntax	Only needed when installing locally; replaced by the information present on http://atoms.scilab.org when publishing
builder.sce	the main builder file	currently required (may be optional in the final 6.0 release)
loader.sce	the main loader file	no need to create it, it generated at build time by builder.sce
license.txt	license file	optional (but license information should be present in the DESCRIPTION, or on atoms.scilab.org)

Build and load a module

Two scripts are required to create a module: a builder, that will be used to build (package) the toolbox; and a loader, that will be run when the toolbox is loaded in Scilab. Both are Scilab scripts (sce files) and must be respectively "builder.sce" and "loader.sce"

The main builder is used to to build the module:

exec builder.sce

Once built, the module can be loaded in Scilab with the main loader script:

exec loader.sce

Main builder

The main builder script is the builder.sce file in the root directory of the toolbox. It relies on several sub-builders, each of one is responsible of build a component of the module.

Following is the code of a generic builder, which executes sub builders:

mode(-1);
lines(0);
try
 getversion('scilab');
catch
 error(gettext('Scilab 5.0 or more is required.'));
end;
// ====================================================================
if ~with_module('development_tools') then
  error(msprintf(gettext('%s module not installed.'),'development_tools'));
end
// ====================================================================
TOOLBOX_NAME = 'toolbox_skeleton';
TOOLBOX_TITLE = 'Toolbox Skeleton';
// ====================================================================
toolbox_dir = get_absolute_file_path('builder.sce');

tbx_builder_macros(toolbox_dir);
tbx_builder_src(toolbox_dir);
tbx_builder_gateway(toolbox_dir);
tbx_builder_help(toolbox_dir);
tbx_build_loader(TOOLBOX_NAME, toolbox_dir);

clear toolbox_dir TOOLBOX_NAME TOOLBOX_TITLE;
// ====================================================================

Start and quit script

Each toolbox has an initialization script (also known as Start script) and a finalization script (also known as Quit script). These scripts are in the etc directory. Their basename (without the mandatory .start and .quit extensions) sets/is the technical identifier of the module. For example the start and quit script for toolbox_skeleton are toolbox_skeleton.start and toolbox_skeleton.quit, while toolbox_skeleton is the official identifier of the module for the ATOMS system. This identifier is case-sensitive.

These script are automatically called by Scilab, when the related toolbox are started or quit.

The start script is responsible of loading the toolbox. For this it relies on several sub-loaders, each one of them loads a specific component of the toolbox.

For example of toolbox_skeleton, the start script toolbox_skeleton.start calls the gateways and the Java package loaders, like this:

...
exec(path_convert(root_tlbx + "/sci_gateway/loader_gateway.sce");
...
exec(path_convert(root_tlbx + "/src/java/loader.sce");

Main loader

The main loader script is the loader.sce file in the root directory of the toolbox. It is used to manually load the toolbox. As the loading is already implemented in the start script, the main loader script can call that start script to do the job.

For example of toolbox_skeleton, the main loader script code is:

exec(get_absolute_file_path("loader.sce")+"etc\"+"toolbox_skeleton.start");

Macros

By convention the builder included in the macros directory is named buildmacros.sce.

In this example, we consider that the macros directory of the toolbox_skeleton module contains just only one macro, the function scilab_sum (see above script). This function returns s the sum of A + B (A and B are the inputs of the function).

Example of Scilab macro: scilab_sum.sci

function s = scilab_sum(A, B)
  s = A + B;
endfunction

The macros builder

The builder (see below script) creates a library variable from macros stored in the directory macros, and saves it in the file calledlib. The builder file contents is generic, it is executed in two steps:

first step locates buildmacros.sce script (see get_absolute_file_path function help page)
second step builds macros (see tbx_build_macros function help page).

The library generated is called: toolbox_skeletonlib i.e. yourtoolboxnamelib (yourtoolboxname must be 21-characters long max).

Contents of buildmacros.sce file:

tbx_build_macros(TOOLBOX_NAME, get_absolute_file_path('buildmacros.sce'));

clear tbx_build_macros;

The macros loader

The loader loads the library generated by the builder in the directory macros.

Primitives

Source code of a primitive native function is stored in the src directory, in the sub-directory corresponding to the language in which it is implemented ("c", "cpp", "fortran", ...). For each Scilab primitive a gateway function must be created, which the source code is stored in the sci_gateway directory. By convention gateway names begin with "sci_"

C Primitives

We consider a C function csum which returns the sum of two scalars. The name of the corresponding primitive in Scilab is c_sum and the associated gateway name is sci_csum. The both following scripts represent the C code of csum and sci_csum.

The primitive c_sum can be called in Scilab as follows:

--> Y = c_sum(A,B)

Source code of csum.c:

int csum(double *a, double *b, double *c)
{
        *c = *a + *b;
        return 0;
}

Source code of sci_csum.c:

#include "api_scilab.h"

/* ==================================================================== */
extern int csum(double *a,double *b,double *c);
/* ==================================================================== */
int sci_csum(char *fname)
{
    /* error management*/
    SciErr sciErr;
    int iRet = 0;

    /* Variables for first input argument*/
    int* piAddr1 = NULL;
    double dblVal1 = 0;

    /* Variables for second input argument*/
    int* piAddr2 = NULL;
    double dblVal2 = 0;

    /* Variable for return value */
    double dblRetVal = 0;

    /* Check that we have only 2 input parameters */
    CheckInputArgument(pvApiCtx, 2, 2);

    /* Check that we have only 1 output parameter */
    CheckOutputArgument(pvApiCtx, 1, 1);

    /* get first parameter and put in 'dblVal1' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 1, &piAddr1);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr1, &dblVal1);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }

    /* get second parameter and put in 'dblVal2' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 2, &piAddr2);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr2, &dblVal2);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* call csum subroutine */
    csum(&dblVal1, &dblVal2, &dblRetVal);

    /* create a variable on scilab's memory */
    iRet = createScalarDouble(pvApiCtx, nbInputArgument(pvApiCtx) + 1, dblRetVal);
    if(iRet)
    {
        /* If error, no return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* assign new variable to return value*/
    AssignOutputVariable(pvApiCtx, 1) = nbInputArgument(pvApiCtx) + 1;
    return 0;
}

Fortran primitives

We consider an fortran subroutine fsum which returns the sum of two scalars. The name of the corresponding primitive in Scilab is fortran_sum and the associated interface program name is sci_fsum. The both following scripts represent the fortran code of fsum and sci_fsum.

The primitive fortran_sum in can be called in Scilab as follows:

--> Y = fortran_sum(A,B)

Source code of fsum.f:

c =================================
      subroutine fsum(a,b,c)
c =================================
      double precision a,b,c
                        c = a + b
      end
c =================================

Source code of sci_fsum.c (we call a fortran subroutine from a C gateway):

#include "stack-c.h"
/* ==================================================================== */
extern int F2C(fsum)(double *a,double *b,double *c);
/* ==================================================================== */
int sci_fsum(char *fname)
{
    /* error management*/
    SciErr sciErr;
    int iRet = 0;

    /* Variables for first input argument*/
    int* piAddr1 = NULL;
    double dblVal1 = 0;

    /* Variables for second input argument*/
    int* piAddr2 = NULL;
    double dblVal2 = 0;

    /* Variable for return value */
    double dblRetVal = 0;

    /* Check that we have only 2 input parameters */
    CheckInputArgument(pvApiCtx, 2, 2);

    /* Check that we have only 1 output parameter */
    CheckOutputArgument(pvApiCtx, 1, 1);

    /* get first parameter and put in 'dblVal1' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 1, &piAddr1);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr1, &dblVal1);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 0;
    }

    /* get second parameter and put in 'dblVal2' */
    sciErr = getVarAddressFromPosition(pvApiCtx, 2, &piAddr2);
    if(sciErr.iErr)
    {
        printError(&sciErr, 0);
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }
    
    iRet = getScalarDouble(pvApiCtx, piAddr2, &dblVal2);
    if(iRet)
    {
        /* No return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* call fortran fsum subroutine */
    F2C(fsum)(&dblVal1, &dblVal2, &dblRetVal);

    /* create a variable on scilab's memory */
    iRet = createScalarDouble(pvApiCtx, nbInputArgument(pvApiCtx) + 1, dblRetVal);
    if(iRet)
    {
        /* If error, no return variable */ 
        AssignOutputVariable(pvApiCtx, 1) = 0;
        return 1;
    }

    /* assign new variable to return value*/
    AssignOutputVariable(pvApiCtx, 1) = nbInputArgument(pvApiCtx) + 1;
    return 0;
}

Primitives builder

Now the src and the sci_gateway directories contain all necessary files to create the builder (see template below) for the primitive c_sum.

We need to write two builders:

On one hand, in the src/c/ directory, this builder (named builder_c.sce) creates a shared library (see ilib_for_link function help page) corresponding to the C function.
And the other hand, in the sci_gateway/c/ directory, this builder (named builder_gateway_c.sce) creates the new shared libraries to link the compiled C or Fortran new Scilab interface routines and generates a loader (see ilib_build function help page). This loader file calls the addinter function to load dynamically the shared library (See addinter function help page).

Contents of builder_c.sce file:

tbx_build_src(['csum','csub'], ['csum.o','csub.o'], 'c', ..
              get_absolute_file_path('builder_c.sce'));

clear tbx_build_src;

Contents of buildsci_gateway.sce file:

tbx_build_gateway('skeleton_c', ['c_sum','sci_csum';'c_sub','sci_csub'], ['sci_csum.o','sci_csub.o'], ..
                  get_absolute_file_path('builder_gateway_c.sce'), ..
                  ['../../src/c/libcsum']);

clear tbx_build_gateway;

Java packages

As of Scilab version 5.5, toolbox functions can be also implemented in Java, to be precise in Java packages. The mechanism is different than the one used for macros or primitives: to build, load and use Java packages, the Java Scilab API is used (there is no need to write a gateway). But from the toolbox developer's view, the process of building and loading is roughly the same in Java.

Java sources

Let us recall that Java packages are organized in a hierarchical way, each package containing sub-packages. The root package is named with a naming pattern which is usually something like org.company.software.package. Each Java class is located in a directory that respects the package hierarchy.

The Java packages are stored in the src/java directory. For each Java package, the sources are stored in the specific sub-directory tree.

For example let's consider the Java package org.scilab.scilab.toolboxskeleton, containing a Java class Sum. The package source tree in src/java may look like as following:

src/java:  
   tooboxskeleton
      org  
         scilab 
            scilab 
               toolboxskeleton             
                  Sum.java

The Sum class contains a static function sum which returns the sum of two doubles. The code is following:

package org.scilab.scilab.toolboxskeleton;
public class Sum {
    public static double sum(double a, double b) {
        return a + b;
    }
}

Java builder

The source builder script builder_src.sce (in src directory) has to be modified to take in account Java sources.

The function used in this script is tbx_builder_src_lang and run the build of build sources for a list of languages. The Java language has to be added to the list of languages, as following:

   langage_src = ["fortran" "c" "java"];
   path_src = get_absolute_file_path("builder_src.sce");
   tbx_builder_src_lang(langage_src, path_src);

The Java building script is the file builder_java.sce in src/java directory.

The needed command to compile Java sources and to build a JAR file is ilib_build_jar. The script code looks like following:

   package_name = "org.scilab.scilab.toolboxskeleton";
   jar_file_path = fullfile(jar_dir, package_name + ".jar");
   ilib_build_jar(jar_file_path, package_name, fullfile(src_java_dir, "src"));

Where:

src_java_dir is the path of the java source directory
jar_dir is the path of the directory where the JAR file is created.

Java loader

The generated Java loader script is loader.sce in the src/java directory. This script has to be called from the toolbox start script.

Help

The help files are JAR files stored in the jar directory. They are produced from the help source files, which are XML files, and a builder script builder_help.sce, all stored in the help directory.

Creation of XML files

Here is a template which shows you how to write the XML help files. You should just fill the different items for your functions and put them in the help directory.

See https://scilab.gitlab.io/legacy_wiki/howto/scilab_documentation_kit for more information about documentation.

Example of Scilab help file, c_sum.xml:

<?xml version="1.0" encoding="UTF-8"?>
<refentry version="5.0-subset Scilab" xml:id="c_sum" xml:lang="en"
          xmlns="http://docbook.org/ns/docbook"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xmlns:svg="http://www.w3.org/2000/svg"
          xmlns:ns3="http://www.w3.org/1999/xhtml"
          xmlns:mml="http://www.w3.org/1998/Math/MathML"
          xmlns:db="http://docbook.org/ns/docbook">
  <info>
    <pubdate>$LastChangedDate: 2008-03-26 09:50:39 +0100 (mer., 26 mars 2008)$</pubdate>
  </info>

  <refnamediv>
    <refname>c_sum</refname>

    <refpurpose>sum from C</refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <title>Calling Sequence</title>

    <synopsis>a = c_sum(b,c)</synopsis>
  </refsynopsisdiv>

  <refsection>
    <title>Description</title>

    <para>Do a sum.</para>

    <para>Add here a paragraph of the function description </para>
  </refsection>

  <refsection>
    <title>Examples</title>

    <programlisting role="example">c_sum(3,4)</programlisting>
  </refsection>

  <refsection>
    <title>Authors</title>

    <simplelist type="vert">
      <member>YOUR NAME</member>
    </simplelist>
  </refsection>
</refentry>

The help builder

This file creates a file containing all your module documentation so that it can be loaded in Scilab help browser.

Contents of build_help.sce:

help_lang_dir = get_absolute_file_path('build_help.sce');

tbx_build_help(TOOLBOX_TITLE, help_lang_dir);

clear help_lang_dir;

Open the help

To see the help pages about toolbox_skeleton, launch the help browser with:

help

Uploading your toolbox

Once your toolbox can be compiled and loaded, you can upload it to ATOMS:

Archive and compress your toolbox (as .zip or .tar.gz). Do not add the compiled code.
Register and log in to the ATOMS website
Add a new module on the ATOMS website, and upload your module as source archive.

Scilab Enterprises can then take care of compiling your toolbox for multiple platforms and versions of Scilab. However, in order to be part included in the ATOMS packaging system, your toolbox must respect the format of the toolbox outlined above. See also the Guidelines on submitting a module below for a few tips.

Q&A

How can I have my module included to ATOMS?

See Uploading your toolbox section above.

Once it is done, we should process (generate binary versions) your module within a few days. If this does not happen, feel free to contact us on the mailing lists, or directly via email.

Note : if you are posting your sources, don't forget to set its class as "sources of a valid Scilab package".

When we have generated binaries for the toolbox, you will receive an automated email; and your module will be available for all users through ATOMS.

To see the new package in Scilab, update your package list by running:

atomsSystemUpdate()

For more details, please read atomsSystemUpdate

What are the technical expectations ?

See above for the requirements. As a reminder:

Use the toolbox skeleton provided, which you can find in the "contrib/scilab_skeleton" of your installation of Scilab:
- On Windows, in C:\Program Files\scilab-6.x.y\contrib\toolbox_skeleton
- On Mac, in /Applications/scilab-6.x.y.app/Contents/MacOS/share/scilab/contrib/toolbox_skeleton
- For releases before the 6.0.0, use the template found on the ATOMS website
You can remove what you do not need from the skeleton.
Edit the files to replace the "toolbox_skeleton" string with the unique name for your toolbox, and "Toolbox Skeleton" with a title (human-friendly name).
If your module only contains macros (Scilab scripts), then the sci_gateway and src folders are not necessary.
A few files are mandatory:
- builder.sce (to build the module) and DESCRIPTION file (formatted in the same way as toolbox_skeleton) at the root.
- etc/<module name>.start and etc/<module name>.quit in etc folder (to load the module).
readme.txt, license.txt, changelog.txt are not mandatory, but are strongly suggested.
Help pages and tests are not mandatory, but are most welcome.
The functionalities you want to expose must be available as functions (.sci with a function in it -- not .sce calling straight the feature)

Note that the content of the DESCRIPTION file will NOT be used on the Atoms website: instead, you must enter all the needed information on the Atoms website itself. However, it is a good idea to enter some minimal information about your toolbox in it.

If you work depends on a third party native library, let us know and please provide as much as information as possible. If you don't want your module to be packaged under a certain OS, please let us know after posting it.

What kind of restrictions is there?

There is no restriction on the kind of modules which are accepted into ATOMS as long as they provide a new Scilab feature (whatever it is).

Since we need to build the module, sources must be available. However, there is no restriction on the license of a module (even if we prefer free licenses).

In any case, do not submit binary files, as they are often not working on different OS or configurations.

Will you modify my module?

Sometimes, we have to patch to make sure that the module builds and runs perfectly. We usually send back the patches for upstream incorporation. Those change are usually :

wrapping your module in a folder named "yourmodulename"
adding a yourmodulename.quit in etc folder
fixing typo/case sensitive problems for building/loading the module (mostly in builder.sce, yourmodulename.start, yourmodulename.quit)

Consider those points before submitting a module, as they are quickly fixed by you and often make us lose a lot of time, which could be used improving ATOMS and fixing its bugs!

I have just a Scilab function/macro and I would like to have it into ATOMS

No problem, just submit it

Preparing for Scilab 6

See Gateways: from Scilab 5 to Scilab 6 for migrating your toolbox to Scilab 6. In particular, stack-c.h is not available anymore in Scilab 6; it should be replaced by api_scilab.h.

Guidelines on submitting a module

clean your package : do not include binary files (.bin, .lib, .dif, .o, .obj files).
create tests for your package : this way, using atomsTest, we can check much quickier if it works on every platform.
name you .zip or .tar.gz as nameoftoolbox-x.x-src.zip or nameoftoolbox-x.x-src.tar.gz, where x.x represent the version number.
create a folder, in the zip/tar.gz file, named nameoftoolbox, in which you put everything.
while submitting, declare it as sources of a package, not as simple file, if you want it to be compiled by our compilation chain.
make it coherent : if you named it with special uppercases on atoms, name it with special uppercases in the folders names and in the builders files, at least.
be sure to have a yourtoolbox.start AND yourtoolbox.quit, and that they respect the uppercases of your module. It should match the url of your module, for example http://atoms.scilab.org/toolboxes/Cardiovascular will have a Cardiovascular.start, Cardiovascular.quit, and a TOOLBOX_NAME Cardiovascular, and not CardioVascular.