[Contents] [TitleIndex] [WordIndex

Contributor - ATOMS Portal


The https://atoms.scilab.org ATOMS portal now homes around 200 external modules (2016). Yet, it still misses many useful or even mandatory features, and have pitfalls. Most of them apply as well for the FileExchange Portal (&FE).

The following bugs are examples of such shortcomings:

Here's also a list of the tasks that we (Scilab Enterprises) have defined, or have started working on. These tasks could get some help too:



Ensure that Atoms builds are done in the same environments as the builds of Scilab

Currently we have specific machines for the builds, but they are not exact clones of the build machines. They should be; this is about setting up (VirtualBox / vagrant) VMs to run the builds.

Remove the "ATOMS packaging system" field, it's useless

- or at least give a better description - it doesn't mean anything: ATOMS packaging system -> "Packages availability" et "This module aims at being included into the ATOMS packaging system ? -> "Should Scilab Enteprises packages this toolbox ?" this is 13819

Prevent renaming files, which is causing weird errors, and forcing the user the do a AtomsSystemUpdate()

when a file is renamed on the server, and a new file is created with the same (previous) name, it causes an error when any user install the the toolbox's renamed file, or any toolbox that depends on this file. The only workaroud is for the user to do an AtomsSystemUpdate(). This is because suddenly the MD5 sums on the local files don't match the MD5 sums from the server. It should **not** be possible to rename files.

Integrate the DESCRIPTION from server when building (on the server)

Currently: in the Toolbox Skeleton, there is a DESCRIPTION file to be filed. However, this information is also on the website. The information on the website SHOULD be the one used; and the DESCRIPTION file from the skeleton is only to be used for toolboxes not published on the website. So, the last thing that needs to be changed, is to ensure that when uploading a binary (or when downloading a source file ?), the DESCRIPTION file is **replaced** on the server, by the file generated by the server (based on the info in the DB). Not a perfect solution but the only correct one to avoid this information duplication issue

Remove the "t" element of the "x.y.x-t" versions

The version after the "-" is currently a workaround to allow multiple sets files to be released for one release. It should not be displayed in the version on the website (instead, it should be displayd as a version of the file, most likely) This is also the "PackagingVersion" field in the DESCRIPTION file generated on the server - but it's barely (and useless) in the Scilab client. This should be cleaned and removed. this would mean present it differently on the website ; and do not use it on the client in the version dependencies (always use the biggest one)

Get the server to only send the latest version for each toolbox

Currently, the server (in the TOOLBOXES.gz files) sends a list of all valid toolboxes for the version of Scilab. Most likely (to be verified) this list could be much shorter, by only sending information about the latest version of each toolbox. This would provide a big performance improvement, which is not yet a prb on scilab 6, but will be relatively soon; and is already one on 5.5. It would also alow simplifying code in Scilab (see next itesm)

Also, get the server to verify that dependencies are handled **in the TOOLBOXES.gz file**

Currently, there is no check that dependencies are correct in the TOOLBOXES.gz file (i.e. that all toolboxes needed as dependencies for all toolboxes are present): this is only done by Scilab once it has received the file. This could be done once for good when creating the TOOLBOXES.gz, and the removed from the client.

Once the two previous items are done, remove useless code in Scilab

simplify dependency handing in Scilab (remove tree-building code). Use the fact that the server only sends the latest version to simplify that code. This would mean, however, that local (non-downloaded from atoms.scilab.org) versions of dependencies would not be handled: but this is not a usual case, and would be OK (to replace by warnings)

Once the three previous items are done, simplify massively the code to handle the dependencies in Scilab

in particular remove support for "<=", ">", etc.. ranges, which are not used (possibly non-compatible, i.e. only on 6.0)

Simplify the build scripts for atoms.scilab.org

This has been mostly done, but need more testing. In particular, be clear on whether it goes to the server to get the DESCRIPTION or not; clarify the options for that.

Better presentation for 5.5 vs. 6.0 on the website: to design.

Currently need two versions (5.5. and 6) for each toolbox; would be better to use the same, when possible (knowing that it's not always possible). Probably means a change in DB design (and a data migration); because of that, probably to keep to a minimum (because migrating data and app is always complex)

Don't show old versions of old toolboxes so proeminently on the website

On the list on the left side, all versions of all toolboxes appear. This makes it a very long list, with mostly irrelevant information. Probably this list should only only present the latest one(s), or one for 5.5. and one for 6.0. We could in fact entirely remove the older versions.

Add ability to post comment without rating, and rating without comment

it doesn't make sense currently to be required the user to add **both**, probably we are loosing a lot of comments (and rating) because of it

Set up Atoms server for tests only

Currently atoms.scilab.org include toolboxes used for tests, which is very weird to the user. The easiest solution would be to set up a specific website 'tests.atoms.scilab.org' for the tests, with its specific DB, and use that for tests. It would also require editing the tests to use this new server. That would be a simple solution (setting up a new atoms site should be easy - we already did it for dev.atoms.scilab.org) this is 14246 mentioned above

A way to publish the toolboxes' help on the website (reported by S. Gougeon)

This was probably what the DESCRIPTION_FUNCTION had been done for, but it was not used anywhere (neither on the atoms.scilab.org website nor in scilab), so totally useless - so I recently removed it from the toolbox_skeletton, and removed any mention of it. But still, would be really nice for the doc to be published (automatically ?)

Remove or fix list of files, and build logs

List of files (as well as tree - but it's not working) and build, and test logs, are kept with the files. This is not very useful (end-users shouldn't need to look at build logs...) and could be removed. Removed build logs, and file tree; kept file list when it is present, and keep it in the build process

Security: risk of SQL injection on atoms.scilab.org

I checked a little, and there is no *obvious* risk - I couldn't find a way to actually modify data on our server though SQL injection (because we ony use mysql_query which only allows one query at at time) nor access user data (they are not stored in mysql but on the LDAP server). However, the site should use prepared statement everywhere http://php.net/manual/en/pdo.prepared-statements.php - shouldn't be complex to change it, just some work

TOOLBOXES.gz are getting regenerated

this is for the 5.5. files in /home/web/fileexchange.scilab.org/5.5 on the server: there's some dup data, probably, that are causing them to be regenated all the time. the prb is: "sourceName: number--1-src.zip" vs. "sourceName: number-1.3-1-src.zip". Probably some dup data fixed

Move rarely - or unused - Atoms functions from Scilab into a specific, optional, toolboxes.

Most users only use very few functions from Atoms: atomsInstall, atomsList, atomsRemove... Many other functions for Atoms currently in Scilab are not useful to the end-user, only used for Scilab Atoms build; in particular: atomsDepTreeShow ; atomsTest (used only in atomsCheckModule - we don't use it anymore for our builds) ; atomsCheckModule (probably not used by anyone);... and probably many more. These functions could be removed, or at least moved to an toolbox that people would only install when needed. It would make Atoms much easier to maintain (for us), much easier to use (for the user).

Create Jenkins jobs to run the builds

the old build website http://nightly.scilab.org/ hasn't been working for a while. Instead of fixing it, we should set up Jenkins tasks on http://build.scilab.org to build Atoms toolboxes; it should be easy, although it may require modifying the build chain to ensure that the output is kept properly.

Check load time and performance

Some pages - for example https://atoms.scilab.org/toolboxes/Mathieu - seem to take a long time to load. We should see if SQL or page is inefficient (missing SQL indexes ?)

Clean up the wiki pages

There are two pages explaining how to create/build toolboxes: https://scilab.gitlab.io/legacy_wiki/ATOMS?highlight=%28atoms%29#ATOMS_for_developers, et il y a https://scilab.gitlab.io/legacy_wiki/howto/Create%20a%20toolbox . They should be at least linked together; if possible clarified; and ideally merged. (this should be a small task). There is also a private page ( https://scilab.gitlab.io/legacy_wiki_project/Compiler%20une%20toolbox%20dans%20Atoms ) which should be made public.

Release the AtomCC toolchain to the community

Internally, Scilab Enterprises has a toolchain, written in Perl, to build and release the binaries for toolboxes. There is no reason for this toolchain not to be public, in fact to help toolbox developers to tests their toolboxes, and even release the (all ?) binaries to atoms.scilab.org . Currently the toolchain requires SQL and ssh access to http://atoms.scilab.org to release the files, but it should be easy to replace that to make it an HTTP upload - that toolbox maintainer could run with their own maintainer accounts

It should be possible to see **who** released a file

summary says it all :)

Many other tedious bugs are pending. Their full list only for the ATOMS portal is here. The full list including resolved bugs is there

For the FileExchange Portal, it is here. Most of its bug apply as well for the ATOMS portal.

2022-09-08 09:26