Getting started

Motivation

Developing computational models can be quite a mess in terms of the file and configuration management. Therefore most of the (climate) models are accompanied with some kind of framework to guide the user through their piece of software. Those however can be very complex and every model follows it’s own strategy. Therefore this package tries to organize the procedure focusing at the end-user by providing an outer framework that allows the user to interfere with the model from the command line.

How does the package work

When doing research, we usually have a specific (funding) project that requests multiple runs (experiments) of our model. The framework of the model-organization package mirrors this basic structure: It works with projects, each project contains several experiments. The package keeps track of your projects and experiments and saves the configuration in separate configuration (.yml) files.

Configuration files

All the paths to the projects are stored in the configuration directory determined by the name attribute of your model (see the config.get_configdir() function). By default, it’s (on linux and mac) "~/.config/<name>", but you can determine it via the global <NAME>CONFIGDIR variable, where <name> must be replaced by the ModelOrganizer.name of your model. Our example below would store the configuration files in "$HOME/.config/square" and the corresponding environment variable is SQUARECONFIGDIR.

The above directory contains 3 files:

globals.yml

The global configuration that should be applicable to all projects

projects.yml

A mapping from project name to project directory

experiments.yml

The list of experiments

Additional, each project directory contains a '.project' directory where each experiment is represented by one yaml file (e.g. 'sine.yml' in our example) and the project configuration is stored in '.project/.project.yml'. To get the specific name of the configuration file, you can also use the exp_path parameter of the info() method or the command model info respectively.

Creating your own ModelOrganizer

The ModelOrganizer class is designed for subclassing in order to fit to your specific model. See the incode documentation of the ModelOrganizer for more information.

Using the command line argument

Each method that is listed in the commands attribute is implemented as a subparser for the for the main command line utility (see our Starting example: square). If you subclassed the model organizer, you can use the main() method to run your model. You can do (for example) as we did in the example via:

if __name__ == '__main__':
    SquareModelOrganizer.main()

You can see, the results of this methodology in the Command Line API Reference.

Parallel usage

The usage of configuration files includes some limitations because the configuration cannot be accessed in parallel. Hence, you should not setup projects and initialize experiments in parallel. The same for the archiving method. However, the run, postproc and preproc method as we implemented in in our example, could be used in parallel.