The Document Upload Process Has Encountered an Issue With Your Netbadge Credentials.
New Results
modelRxiv: A platform for the distribution, computation and interactive display of models
Abstruse
Modeling the dynamics of biological processes is ubiquitous across the ecological and evolutionary disciplines. However, the increasing complexity of these models poses a meaning challenge to the dissemination of model-derived results. With the existing standards of scientific publishing, most often only a minor subset of model results are generated, presented in static figures or tables, and made available to the scientific community. Further exploration of the parameter space of a model, investigation of possible variations of a model, and validation of the results in relation to model assumptions commonly rely on local deployment of code supplied by the authors. While releasing model code is a publication requirement for most scientific journals, at that place are currently no standardized protocols or coding-language requirements. Deploying models locally poses a technical challenge due to the specific framework and surroundings in which a model was developed, and tin preclude model validation and exploration by readers and reviewers. To address this issue, we developed a platform that serves as an interactive repository of biological models, chosen modelRxiv (https://modelrxiv.org). The platform provides a unified interface for the analysis of models developed in multiple programming languages but does not require whatsoever technical agreement of the model implementation. To reduce model computation time, the platform allows users to pool computational resources available to them, including lab workstations, clusters and cloud services. Past making published models accessible, this platform promises to significantly improve the accessibility, reproducibility and validation of ecological and evolutionary models.
ane Introduction
Modeling the dynamics of ecological and evolutionary processes, as well as of other biological processes, has been cardinal to the development of ecology and evolution for more than than a century [ane]. Over the past few decades, the availability of computational power has paved the way for models of increasing complexity, involving numeric, stochastic, and individual-based modeling features, and exploration of multiple parameters spanning over large parameter spaces [2]. Detailed and high-resolution investigation of models can offer important insights on full general phenomena likewise as on specific systems; nonetheless, traditional scientific publishing standards—static papers depicting several model results representing select parameter values—are limited in demonstrating the full scope of the model results and their implications. This affects both the dissemination of potential insights from a model and the power to assess the robustness of model results during the scientific peer-review procedure.
Currently, models in ecology and evolution, and across the biological disciplines, are implemented in a number of different programming languages, and there is no standard framework for publishing model code. Consequently, deploying models locally by a reviewer or a reader usually requires prior knowledge of the specific coding language or framework in which the model code was written, the installation of dependencies, and learning how to operate the model code and visualize results. This technical hurdle tin make published models essentially inaccessible, where only the authors of the model are able to reproduce its results.
Several platforms have been adult to address some of these difficulties. These approaches can be categorized into ii wide classes: (i) a 'model-centric' approach, where the emphasis is on model analysis and visualization, and users are not required to interact with the underlying model lawmaking, and (two) a 'code-centric' approach, where users collaborate with model lawmaking in order to analyze the model. Model-centric platforms provide a single programming language for modeling (eastward.one thousand., NetLogo [three], Mathematica [4]), while code-axial platforms accommodate the deployment of multiple programming languages in a unmarried interface (eastward.g., Jupyter [5], CodeOcean [6]). In both classes, models are accessed through a unified interface, reducing the investment needed past reviewers or readers to interact with the model. However, most published models are implemented in different programming languages, and translating model code so that it is compatible with a model-centric platform can be impractical. On the other hand, lawmaking-centric solutions that support multiple programming languages are focused on model implementation, and lack features that can make model analysis straightforward and accessible.
To bridge the gaps betwixt these two approaches, we adult an interactive repository of biological models called modelRxiv (https://modelrxiv.org). Our browser-based platform provides a solution to the technical challenges involved in analyzing published models, and is designed to provide a model-centric solution while supporting multiple programming languages and frameworks. In addition, to support the increasing amount of computational resources needed to clarify models, the platform allows distribution of model computation across multiple deployment environments. modelRxiv is designed with a simple user interface that does not crave a technical understanding of programming or modeling to operate. We envision this platform every bit a tool for model validation and exploration by reviewers and readers, and equally a repository that can make published and unpublished models accessible to the scientific community.
2 Platform
modelRxiv is a browser-based awarding, with model visualization occurring in the browser, and model computation occurring in the browser or in boosted environments (see section 2.3). The modelRxiv user interface has three main screens: (i) an index of models, which can be filtered and sorted co-ordinate to model metadata such as title, authors, model categories, and keywords (Fig. 1); (2) model analysis pages, which include model visualization, parameter manipulation and model analysis features (Figs. 2-4); (3) a model builder page for developing and uploading models. Models can be either 'public', viewable by everyone, or in 'sandbox mode' where merely the owner can see and collaborate with the model. Public models tin can be viewed and analyzed without logging in or creating a user, but registration is necessary to upload models and clarify models in sandbox mode (registration is free and anonymous). In this section we describe the master features of modelRxiv, which can be accessed through the model analysis folio or the model builder page.
- Download figure
- Open in new tab
Figure 1: 'Alphabetize of models' page of modelRxiv, with listing of models belonging to dissimilar categories.
On this folio users tin can search and sort models, and select a model to view its 'model analysis page'. When authenticated, the index will list models that are publicly available, every bit well as private models uploaded by the user.
- Download figure
- Open up in new tab
Effigy 2: The 'model analysis folio' of a hawk-dove model, demonstrating different display types and parameter manipulation.
This page shows the basic display of a model while running. On the left are four panels that show the dynamics in three dissimilar approaches to the militarist-dove model (the fourth panel shows the spatial structure of the interacting particle organisation). These dynamics relate to the parameters selected in the parameter console on the correct.
- Download figure
- Open up in new tab
Figure iii: An 'output summary plot' for the hawk-dove models.
The iii panels on the left evidence output summary plots produced for dissimilar regions of the model parameter space. Colors in these panels relate to outcomes of the model for the specified input parameters: all approaches agree (blue), interacting particle arrangement is different and the non-spatial approaches agree (green), detached and infinite population approaches differ (cyan), and all approaches disagree (ruby). The parameter bill of fare on the right has two parameters selected with a range of values (a and b in the range [0, i], c and d with fixed values); clicking the 'Grid' button produces the plots on the left. In this example, the intensive computation needed to generate the summary plots was conducted by distributed computation on the three unlike machines that appear in the resources menu on the top right, denoted by labels: a local motorcar connected via the browser (labeled 'Home'), a remote machine connected via the browser (labeled 'Laptop') and a lab machine connected via a back-cease utility (labeled 'Lab'). For each machine, a listing of supported languages is also given: the two machines connected via the browser ('Home' and 'Laptop') tin can process JavaScript ('.js' label) and Python ('.py' characterization) code; the machine connected via the back-end utility ('Lab') tin can besides process R files. The resource card also lists the number of computation threads that can be used for each machine. To control ciphering distribution, the computation threads can be adapted by changing the number in the boxes.
- Download figure
- Open up in new tab
Effigy 4: Using 'parameter presets' to recreate figures from published papers.
We define a 'deployment outcome' equally a model output, and assign colors for each possible outcome (colors hither are the same every bit in [8]). The presets appear on the right below the model parameters (e.g., 'spillover dynamics', 'failure dynamics', etc.). On the left we see the dynamics of 1 outcome of the model, 'differential targeting', and in the second and tertiary panel we reproduce the event of two panels from Figure 2 in the original manuscript.
two.i Visualization of model dynamics
To demonstrate the visualization options available on modelRxiv, nosotros implemented and analyzed a well-known gear up of models in ecology [7]. The original manuscript [7] included 4 different approaches to modeling predator-prey ('hawks' and 'doves') competition: (i) a dynamical system with infinite and continuous population sizes and no spatial component, (ii) a reaction-diffusion arrangement with continuous space, (three) a random patch model with discrete individuals and no spatial structure, and (iv) an interacting particle system with detached individuals and spatial construction. The paper aimed to demonstrate the importance of modeling assumptions of spatialness and discreteness in ecological modeling [7]. We implemented (i), (iii), and (iv) in JavaScript and uploaded them to modelRxiv as a single model.
In the model analysis folio, model dynamics can be produced "on-the-fly", where visualization of the model occurs in parallel to the computation. This can be done when the model provides a recurrence function that returns the step t + one of the model given stride t, and when the model uses a framework that is browser compatible (Table 3). This type of visualization allows users to break and restart model computation while the model is running, as dynamics are generated. Alternatively, models tin can return the dynamics of the model upon completion. The platform can display whatsoever parameters returned by the model functions (run across Methods), co-ordinate to the definition of the parameters by the model owner (run across section 2.5 for an explanation on how parameters are defined).
Table 1: Features of modelRxiv compared to other platforms for modeling.
Plus signs indicate that a feature is an integral function of a platform, while minus signs indicate that the characteristic is not available. Where the feature can be made available with additional software or through special features, the name of the software or feature is indicated. For EBI BioModels, which does not serve as a sandbox, distributed computation is irrelevant.
Table 2: Functions that institute model code.
Model code is organized so that its components are composable: defaults return a parameter set that tin exist passed to run or step, stride output tin can be passed to footstep (recursion) and multiple steps tin be passed to result, which has the same output as run. Model lawmaking must include defaults and run; step-wise models must also include pace and result in order to permit dynamics to be generated "on-the-fly" in the browser.
Tabular array 3: Programming languages that can currently be used with modelRxiv in browser and dorsum-end environments.
Plus signs indicate that a language is fully implemented while minus signs indicate that a language is not available. Where a language is available through third-party software, the name of the software packet is indicated.
For the hawk-dove models nosotros implemented, the visualization includes four panels, iii depicting the average population size of hawks and doves in the three models, and 1 depicting the state of the interacting particle organisation model in a 2-dimensional filigree (Fig. two). To the right of the four panels is a carte du jour for manipulating the model parameters (box on the far right of Figure 2). This carte du jour offers two options, either irresolute the model parameters manually (hither we display only a, b, c and d, which are the parameters determining the predator-prey dynamics), or choosing parameter presets (discussed in the iv department below).
two.2 Analysis of models
Visualizing dynamics with stock-still parameter values, such as in Effigy ii, tin can be informative for understanding the dynamics in a given scenario, just ofttimes there is a need to explore the beliefs of models over a large part of the parameter space. We added a feature in modelRxiv to support such investigations, which visualizes model outputs for a range of input parameters in the form of a filigree. We refer to these plots equally 'output summary plots', every bit they visualize the outputs of multiple model runs. For the above hawk-dove model, we define an output parameter, termed the 'outcome', as the agreement or disagreement betwixt the dynamical system and the random patches or interacting particle system. To demonstrate the generation of an output summary plot for this model, we define iv possible values of the event parameter in terms of potential predator-prey co-existence (Fig. 3): all approaches agree (in blue), the interacting particle organization is different and the other two approaches concur (in green), discrete and infinite population approaches differ (in cyan) and all approaches disagree (in cherry-red); these modeling arroyo-agreement outcomes were the focus of the original manuscript ([7]). The values and colors of the outcomes can be defined in the model builder page (run across department 2.5). With these outcomes divers by the model possessor, users can select a range of parameter values for investigation for 2 parameters, and fix the values of the remaining parameters. Past selecting the 'Grid' push button, an output summary plot of the selected parameter infinite is computed and displayed (Fig. 3). These summary plots are an important aspect of model analysis, equally they permit more comprehensive exploration of the parameter infinite of the model compared to running dynamics for specific parameters.
2.3 Distribution of ciphering
For many modernistic models, a comprehensive analysis requires large computational resource that necessitate distribution of ciphering through multi-threading. However, multi-threading can present technical difficulties that preclude many readers and reviewers from attempting to investigate models. To accost this difficulty, we integrated uncomplicated and user-friendly features for distribution of computation directly in modelRxiv. To achieve this, we use a characteristic for multi-threading available in most browsers (see Methods). This allows modelRxiv to distribute grid ciphering across multiple threads of the local machine as well as to pool computation from several continued machines. This can be useful, for example, when users wish to generate high-resolution output summary plots (e.grand. Fig. 3), and requires little-to-no technical knowledge from the user.
Terms and definitions
To illustrate the pregnant of the terms with respect to a specific model, nosotros refer below to the model of gene drive spread described in section 2.four.
Model input parameters. A set of parameters that can exist manipulated by the user, that are received by the model functions. Input parameters take default values that are defined by the model owner. In the gene bulldoze spread model, an input parameter could exist the migration rate between populations, or the initial frequency of the factor drive allele.
Model dynamics. A set of parameters returned for each step of the model that can be visualized while the model is running or when it has completed. A dynamics parameter in the gene bulldoze spread model could rail the frequency of the gene drive allele over time in each population.
Model output. A gear up of parameters returned for each model run. An output parameter in the gene drive spread model could be the deployment outcome, determined in the model lawmaking co-ordinate to the frequency of the factor drive allele in each population in the final model step.
Dynamics plot. A plot that is produced either while the model is running, or later on it has completed, using 1 or more dynamics parameters. The dynamics plots generated for model dynamics depend on the definition of dynamics parameters by the model owner (due east.g. continuous or discrete).
Model output summary plot. A plot that is produced when analyzing a model using ranges of input parameters. The type of plot volition depend on the input parameters selected and their definition by the model owner.
In the multi-threading feature, individual machines connect to a messaging server that is function of the modelRxiv platform (see Methods), and batches are distributed to other machines endemic past the same user, as sets of parameters. When the batches are processed, the results are returned through the messaging server. The procedure of generating grids using the browser-based interface using multiple machines is identical, but tin reduce the computation fourth dimension of the grid (eastward.g., the panels in Figure 3 were distributed beyond three machines). Thread usage can exist managed via the resource tab, by changing the number of utilized threads on each resource (Figure three, meridian right corner).
Machines can exist connected to modelRxiv in two ways. I option is logging into the browser-based interface on modelRxiv with the same user business relationship from different machines. After clicking the 'Connect' button in the resources menu, resources from the connected machines will exist pooled. This choice allows straightforward replication of the browser surroundings without requiring the installation of software on the connected machines. To puddle resources through other environments, it is possible to connect to modelRxiv via a back-end utility that is available as part of the platform code. This allows users to compute models using the local environment on the connected auto, and so that frameworks installed on this machine can be used. This offers much more flexibility than the browser environment in terms of support for programming languages and frameworks (for details on deployment to multiple environments, run across Methods).
2.iv Reproducing figures using parameter presets
To maintain continuity between a manuscript and an equivalent model on modelRxiv, the platform allows the definition of 'parameter presets' that announced alongside the model every bit part of the parameter panel (Fig. 4). Presets are an important attribute of making models accessible, every bit they direct users to specific sets of parameters that are of interest, or that have been referred to in the manuscript. For reviewers, presets allow the reviewer to reproduce dynamics that relate to specific figures or results, and to manipulate other parameters to test the robustness of the results described by the authors.
To demonstrate this characteristic, we implemented an eco-evolutionary model that tracks gene drive (a genetic element that violates Mendelian inheritance) spread in a ii-population system [8]. We chose this model because it combines evolutionary (the spread of the gene drive) and ecological (migration) elements. The model receives as input the gene drive pattern in the form of three parameters (s, the fettle price of the cistron drive allele; h the dominance of the gene drive allele; c the conversion rate from heterozygotes to gene drive homozygotes), and migration rate between the two populations (m). The model tracks the frequency of the gene drive allele in the two populations (q ane and q 2). The model returns as an output parameter the eventual 'deployment outcome' of the gene drive (Fig. 4): loss of the gene drive allele in both populations (q 1 = q 2 =0, in bluish), fixation in both populations (q ane = q two = 1, in red), or 'differential targeting', where the allele has a high frequency in 1 population and a depression frequency in the other (q 1 > q two in yellow and q 2 > q 1 in green). These outcomes permit investigation of how different gene drive designs are expected to generate dissimilar deployment outcomes, in relation to the connectivity between the two populations.
The paper nosotros modeled [eight] describes a partial exploration of the four-parameter space (s, h, c, and m), with unlike figures describing different features of the dynamics. To enable reviewers and readers to navigate the connection between the newspaper and the model implemented in modelRxiv, we defined presents that reproduce results in the paper. For case, we tin ascertain parameters that volition display the model dynamics for different deployment outcomes, every bit a means of illustrating how the dynamics of a particular outcome are characterized. In addition, we reproduce the parameters of ii panels from one of the figures (in Figure 4, these appear every bit "Fig. two"). In the model assay page, the beginning panel shows the dynamics of each population for the differential targeting issue ('DTE'), while the second and third panel demonstrate the relationship betwixt the initial frequencies of the gene drive and the deployment event, for unlike migration rates. Past clicking on the presets, users generate the output summary plots from the paper; they can also modify the parameters and run custom plots derived from the original analyses. To connect dynamics visualization with grid display, users can click on any point in the filigree to run the model for that specific parameter prepare.
2.v Uploading models
Uploaded models can be written in multiple programming languages. This is adamant by the environments available to the user running the model (see Methods), and linguistic communication-specific wrappers that communicate between the platform and the model code. For the initial release of modelRxiv we provide three wrappers for JavaScript, Python and R. Otherwise, the upload process is independent of the linguistic communication used by the model.
To streamline the process of uploading models to modelRxiv, we developed an upload interface that includes testing and output validation. The upload form is accessible to authenticated users by clicking the 'gear' icon at the top correct corner of the screen and selecting 'Upload model'. The interface is designed as an analog to interfaces for submitting manuscripts for peer review. The upload course is designed to ensure that models operate equally expected before deployment. The procedure is separated into unlike parts of the upload form: (i) adding metadata associated with the model, such as the title and author list; (ii) calculation model lawmaking and testing the code for syntax errors or integration bug; (three) defining the types of inputs and outputs of the model. modelRxiv volition endeavour to place the inputs and outputs based on the model code; after, users tin label these parameters manually. As an culling to providing model code in a specific programming language, users tin can generate lawmaking from equations that describe the model using a simplified model builder (see the 2.half dozen section).
Upon uploading (by clicking the 'Submit' button at the end of the upload form), models are visible and accessible only to the account owner, and appear equally 'sandbox' models. This gives users the ability to ensure that the model operates correctly before submitting it to the public modelRxiv repository. Uploaded models tin exist edited using the 'Edit' button on the model page. Model lawmaking and parameters are versioned, so that previous versions tin be browsed and restored if necessary. To submit the model to the public repository, users click the 'Publish' push button on the model analysis page. This will transfer the model to a moderator who will review the model manually to ensure it does not comprise malicious code, and later the model volition be publicly available on modelRxiv.
In the model analysis folio, plots will exist generated automatically based on the blazon of the output parameters. For instance, continuous variables returned by the footstep function will be translated into a line plot, with the model steps as the x-axis, and the value of the variable as the y-axis, whereas an array of two-dimensional vectors would exist represented as a scatter plot that is updated at every stride. It is likewise possible to define units for output parameters; parameters with the aforementioned units and domain can be grouped into the same plot (eastward.g., multiple line plots can exist combined into a single plot with dissimilar line colors, equally in the get-go panel of Figure 4).
2.half dozen Simplified model builder for mathematical models
In some cases, models tin can be explicitly described equally a set up of mathematical equations. To facilitate adding such "computationally simple" models to modelRxiv, nosotros developed a 'simplified model builder' for edifice models past providing equations in a Python-like pseudocode. This utility is geared towards models that can exist defined as a step function (e.one thousand., detached-time models), just information technology could be extended in the future to other types of mathematical formulations.
The main component of the simplified model architect represents a unmarried iteration of the model dynamics, such as a single time stride (Fig. half dozen). First, the user defines the parameters of the model and their default values. Currently, any variable whose definition does non include other parameters volition be interpreted as an input parameter. Next, the user can ascertain a step function as a series of equations, which can include any parameter that has already been defined (e.chiliad., Fig. 6). Finally, the user defines the parameters that are computed at each step of the function. Note that the value of an input parameter that is also returned past the footstep function will be overridden by its new value; for example, the input parameter 'q1' in Fig. vi has a default value defined on line five; this will be the initial value of q 1, and this value will be updated at each step by the equation on line 22. After building the model lawmaking, users tin define the type and domain of input and output parameters in the course sections below the model builder (Fig. v).
- Download figure
- Open up in new tab
Figure v: Interface for uploading models.
The first section contains the model metadata, the 'script' section contains the model code or equations, and the 'model parameters' section defines the types and ranges of input and output parameters of the model. Metadata can be automatically filled out by searching PubMed via the search push button in the title field, while input and output parameter fields are automatically filled out after successfully testing the model code.
- Download figure
- Open in new tab
Figure 6: Pseudocode that volition be interpreted as a model step role for cistron bulldoze spread model.
Python-style comments denote different parts of the model: (i) default input parameter values; (2) equations composing the step function; and (3) a return statement that defines the outputs of each step.
3 Discussion
Nosotros present and describe the features of the modelRxiv platform, an interactive repository of models. The purpose of the platform is to facilitate reviewers and readers in evaluating and investigating models, and to increase the accessibility of published models in environmental and evolution. In addition, the platform can be used as a sandbox for designing and developing models. The platform is designed as a unified spider web-based interface for working with models, while having the flexibility of running models written using multiple programming languages, and allowing distribution of model computation across multiple environments. In order for the platform to be successful in improving model accessibility and review processes, it would need to be widely adopted by the members of the scientific ecological and evolutionary modeling customs. Nosotros have, therefore, designed this version of modelRxiv with the features that we believe will be useful for experienced modelers and newcomers akin. Future additions and extensions of features will be guided by feedback from the ecological and evolutionary modeling customs. To allow users to provide feedback on bugs and issues using modelRxiv, nosotros have opened a Slack workspace that can be joined using the invitation link on the 'contribute' page (https://modelrxiv.org/contribute).
With evolution of new platforms such as modelRxiv, information technology is important to evaluate and compare the suit of suggested features to existing platforms to amend sympathise its potential contribution. There are a number of established comparable platforms that provide an interface for manipulating model parameters and visualizing dynamics or results, mainly Mathematica, MATLAB, Jupyter and NetLogo. In modelRxiv, we accept aimed to develop a combination of the features of these platforms that will be almost useful for making a wide range of published and unpublished models accessible, summarized in Table 1. On the 1 mitt are 'model-centric' platforms such as NetLogo [three], Numerus [nine], MATLAB [10], or Mathematica [4]. These require that models exist written in a programming language adult specifically for the platform. In principle, such platforms tin can offering a coherent user feel, where the end-user is able to access the aforementioned options equally the developer of the model. However, in practice, most models are still developed in different languages and frameworks, mainly because different users are familiar with unlike coding languages. In improver, only some of these platforms have a web-based implementation. An alternative is 'code-axial' platforms that back up execution of multiple programming languages, such as Jupyter [5]. Jupyter aims to provide a unified wrapper for code that serves both as a development sandbox and equally an interactive interface for deploying code. Even so, because Jupyter is programming-oriented, it emphasizes the implementation of the model rather than its logic and introduces technical hurdles that need to be overcome in society to clarify the model. Thus, a platform that is designed specifically for modeling, but also has the flexibility of supporting multiple programming languages and frameworks, has the potential to provide a coherent, accessible solution for many different modeling approaches.
modelRxiv also aims to implement features of the repository EBI BioModels [11, 12], which provides a browser-based interface for accessing published mathematical models. EBI BioModels is designed mostly for biochemical models, and lacks the power to dispense models from the repository interface. In designing modelRxiv, nosotros implement repository elements like to those on EBI BioModels, and combine these with features specifically suited to analyzing modern eco-evolutionary models, such as parameter manipulation and distribution of computation.
modelRxiv offers both the flexibility of deployment in multiple programming languages, and the accessibility of a user interface that requires no technical understanding of the underlying code. By using presets, the model possessor can guide users through stages of exploring the parametric space of the model. This tin lead to a more than intuitive understanding of the model than a static figure, as the user is gratuitous to manipulate the model parameters and observe the effect on the model result, or to visualize model dynamics for different regions within a sure figure.
3.1 Facilitating evaluation of models during the review procedure
One of the nigh of import potential applications of modelRxiv, in our opinion, is to facilitate and improve the review processes of modeling studies in ecology and evolution. During the review of modeling studies reviewers should, ideally, evaluate the correctness, robustness, reproducibility, and applicability of models. However, in practise, even when the model code is attached to the submission, there are technical difficulties including setup, installation of dependencies, compatibility, and acquaintance with the specific coding language chosen by the writer that makes this time-consuming and unfeasible in many cases. Consequently, models are rarely reproduced by reviewers during the review process, and are even more rarely subjected to manipulation and thorough investigation beyond the specific parameter values chosen by the writer.
Improving model evaluation during the review process requires evolution of user-friendly and accessible tools for reviewers. The understanding that such tools are vital to ensure code validation has led to the adoption of services for deploying data processing code to computational containers by reviewers, such as CodeOcean [half dozen, 13]. CodeOcean provides an interface through which reviewers can manipulate and deploy code associated with a manuscript during the review procedure. By removing the technical hurdles of code deployment, lawmaking validation tin go an integral part of the review process without resulting in a significant additional burden on reviewers.
modelRxiv aims to provide a tool for model evaluation, deployment, and manipulation for eco-evolutionary models. By providing a unified user-interface for models written in dissimilar programming languages, modelRxiv resolves the technical challenge of understanding and deploying model lawmaking. With models uploaded to modelRxiv, reviewers have access to the total parameter space of the model. Using the presets feature, reviewers could easily generate figures from the manuscript, and assess the robustness of the model in terms of the parameters used to generate these figures.
Making model lawmaking validation an integral part of the review process would improve non only confidence in the model results, simply also encourage more than thorough exploration of the model parameter infinite by the authors. It could besides shorten the review process past allowing reviewers to answer questions regarding model parameters without having to rely on the authors to produce boosted figures. Every bit modelRxiv is a public repository, using information technology in the review process has the of import benefit of ensuring that the model would be accessible to readers of the manuscript after it has been published.
3.2 Public accessibility to model results
Across facilitating the review process, nosotros believe that modelRxiv could promote and meliorate the exploration of published models in the ecological and evolutionary disciplines. Kickoff, the ease of accessibility to published models would incentivize researchers to expand existing models and utilise existing modeling frameworks, thereby making model development faster. This would too amend the comparability betwixt published models, ensuring that conclusions pertaining to differences between model results can be coherently attributed to changes in key assumptions, rather than to model design or coding. Second, encouraging the scientific customs to participate in manipulation of model parameters, in a deeper examination of model parameter spaces, and in aligning of model assumptions through simple alteration of underlying code, could generate new insights for existing models. Such inquiries and modifications could lead to the identification of novel model behaviors with biological significance, perhaps undetected due to a different focus of the original written report. These investigations could also atomic number 82 to a deeper understanding of the model behaviors, particularly in terms of the boundaries in which the described behaviors of the models no longer agree, and a discussion on the biological significance of these boundaries. Therefore, adoption of modelRxiv past the eco-evolutionary modeling community could encourage collaborations between researchers to extend and elaborate on modeling studies, for example between the publishers of the original modeling study and the researchers identifying interesting behaviors in their models.
three.iii Educational uses
modelRxiv besides serves as an educational tool. With models that demonstrate basic principles in ecology and evolution, students can visualize pre-prepared dynamics and dispense model parameters to proceeds intuition on the phenomenon in question, and they tin exam hypotheses regarding the relationship between model parameters. At a more avant-garde level, students can explore the lawmaking implementation of the model, to generate similar alternative models and to gain feel in model coding and design. The fact that the user is not exposed to the total complication of the model from the very commencement is an of import aspect when encouraging those not familiar with the model or its implementation to engage in exploration of the model. Therefore, the clear separation of model visualization and manipulation from the underlying code would be helpful in pedagogy environments, where information technology is necessary to business relationship for variable technical abilities of students to provide individualized and flatter learning curves for students.
4 Methods
Here we provide the technical details of model integration with the platform, and considerations relating to the current implementation of the features mentioned in the Platform section. These details are provided equally an explanatory layer for the platform code, and to make integration with and extension of the platform accessible to new users and developers.
iv.i Elements of model code
modelRxiv acts equally a wrapper for existing models. Integration with models relies on the definition of common inputs and outputs of a set of functions that constitute the model (Table two). These inputs and outputs include: (i) parameters, an associative array of model parameters; and (ii) result, an associative array of model effect variables.
The following functions must be implemented by the model code: (i) defaults, which returns the default parameters of the model; (2) run, which returns a model issue for a set of parameters. For step-wise models that tin be run in the browser, the model code should likewise implement the following functions: (3) pace, which returns step t + one given step t and a ready of parameters, and (iv) result, which returns a model result for a series of steps and a prepare of parameters.
This structure reflects dissimilar parts of the model logic. Considering modelRxiv provides standardized visualization based on the definition of model result variables, the model code should include only the model logic, without any plotting functions. In addition, equally modelRxiv tin can also define parameter grids for model analysis, such analyses demand not be coded in the run function.
4.2 Distribution of computation
modelRxiv includes a number of features that are designed to reduce the computation time of output summary plots. This includes distribution of model computation across threads of the local car, and also the ability to puddle resources from multiple machines. This distribution organization is also designed to connect to institution and cloud service resources. With models requiring increasing computational power, information technology is important to include distribution features as part of modeling platforms.
Modern browsers implement a background job system ( Web Workers) [14] to prevent interface interaction existence degraded by computational requirements of web applications, with each 'worker' deployed to a separate thread. modelRxiv uses Web Workers to parallelize model computation, combining model runs into batches and distributing these across multiple threads. In addition, multiple machines can puddle resources by connecting to a messaging server using the WebSocket protocol, which allows bidirectional communication betwixt clients and a central server [15]. Chiefly, WebSocket has native browser and back-end implementations, allowing it to serve equally a cross-platform means of transmitting data.
Past deploying a WebSocket server, nosotros are able to link multiple machines that connect to modelRxiv. This allows a user to distribute the generation of output summary plots requiring extensive computational power on the local machine, and also on any other machines connected to the same user, by sending batches through the WebSocket server. When the private batches have been processed, the results are sent back through the WebSocket server and combined to produce the plot.
4.3 Deployment to unlike environments
At present, browsers accept native support simply for JavaScript. In add-on, the majority of modern browsers back up WebAssembly [16], which allows programs written in C/C++, C# and Rust to be compiled and run natively. This has paved the way for projects such as Pyodide [17], which allows Python and many core scientific Python packages (e.g. SciPy [xviii]) to be run in the browser. Notwithstanding, the browser environment is limited for many reasons (including security considerations that are irrelevant to model computation), differs betwixt browsers, and is optimized for JavaScript.
To enable computation of models written for a wider range of programming languages and frameworks, modelRxiv includes support for distribution of computation to additional environments (e.g., defended servers, institute computation clusters, cloud environments). Whatsoever environs connected via the WebSocket server using the same user account will have its resources pooled and available to use for model computation (model ciphering volition be distributed according to the relevant frameworks available on each environment). Below nosotros draw some examples of environments to which modelRxiv can be deployed.
4.3.1 Back-end utility
To allow users to utilize frameworks that are inaccessible from the browser environment, or to connect machines without the browser-based interface, we provide, as part of the platform code, a back-finish utility that communicates with the WebSocket server merely lacks an interface. Connecting through the back-stop utility volition allow batches to be received and processed by the connected motorcar. As the utility lacks an interface for interacting with models, it does not allow the initiation of analyses. The back-end process utility comes with wrappers written for specific programming languages; for the sake of demonstrating the capabilities of this utility, we provide three initial wrappers for JavaScript, Python and R. Similarly to the browser-based interface, the utility is written in JavaScript but is run using Node.js, and spawns a thread using the wrapper of the respective programming language. Packages installed for Python or R on the machine will as well be available to the utility (this will be the same for whatsoever programming language or framework supported by the utility). When running the dorsum-end utility, users will be prompted for their credentials for modelRxiv equally they would on the browser-based interface; later on authenticating, the resource on the connected motorcar volition go available for use via the browser-based interface (every bit in the resource menu in Effigy 3). For instructions on installing and using the utility, run across the GitHub repository (https://github.com/carrowkeel/modelrxiv/README.dr.).
iv.3.ii Cluster or queue system
By default, the back-finish utility assumes that allocated resource (threads) can be utilized immediately, and runs computation past spawning subprocesses of the utility. In almost academic institutions, ciphering clusters rely on queue systems to manage thread usage. To facilitate integration of modelRxiv with such computational resources, we developed an additional "fashion of operation" of the back-terminate utility that is compatible with Slurm Workload Manager. Integration for additional software used by clusters can be added in hereafter versions past calculation additional modes.
This mode can be activated by setting --manner=slurm when running the back-cease utility (for detailed instructions see https://github.com/carrowkeel/modelrxiv/README.md). In this mode, the back-end utility launched by the user serves just as a 'relay instance' for submitting jobs to Slurm, and does not directly initiate model ciphering. After launching the utility in this mode and hallmark, received batches are submitted as jobs to Slurm using the sbatch command. When the job is candy past Slurm, it launches a new instance of the dorsum-end utility, which distributes computation of the batch among threads allocated to the job. When the batch is candy, the results are returned to the WebSocket server through the new instance of the dorsum-end utility, which is and so terminated, while the 'relay instance' continues to run.
4.4 Integration with deject services
It is also possible to pool resource from deject services using similar schemes to those presented to a higher place. For the sake of demonstrating the potential integration with deject services, we nowadays ii approaches: (i) launching 'on-demand' compute containers, or (2) using managed compute containers. It is important to note that, as opposed to using resource that are free or belong to the aforementioned enquiry institute, using cloud services without close monitoring of usage, and without an intermediate service that manages usage, tin atomic number 82 to chop-chop exceeding budgets set up out for a project. Thus, we offer examples of integration with cloud services only for the sake of users who are already using these services and are enlightened of the risks involved. In addition, to fully integrate this selection as a characteristic in the modelRxiv platform, it would be necessary to consider the relative cost of the dissimilar resource available for computation when distributing batches, and to blueprint fail-safes that forbid unintended use of paid services. These features are not bachelor in the initial version of the platform, and we recommend care in developing any integration with cloud services to avoid unintentional costly deployments.
four.iv.1 Compute containers
A straightforward arroyo to integrating deject services with modelRxiv is to launch compute containers, which are similar to temporary back-end environments. The compute containers can exist launched using a container image that includes the dependencies required by the model code, and are given a fixed timeout to ensure that containers are terminated when no jobs are received (implementation of the termination of the container depends on the cloud provider). In this scheme, a user wanting to increase computational power for a specific analysis launches multiple containers with the same image. Containers can connect using the dorsum-end utility when launched, and volition appear in the resources management tab. In this implementation, the containers would terminate later on all jobs accept been candy.
4.4.2 Managed compute
A more responsive arroyo to scaling computational power is to utilize managed compute containers. These containers are launched only for the duration of batch processing, and tin can ship the results of model computation to the WebSocket server when consummate. The benefit of managed compute is the elimination of deployment time, decreased "cloud waste" owing to idle containers, and more than back-up in case of a specific thread or container failing. These benefits come at a college cost for equivalent computational power and require the development of additional features to prevent unintended use. To distribute model computation using managed compute services, it would exist possible to replicate the current pick for deploying jobs to Slurm to instead launch an instance of the container via a control-line utility on the local motorcar (most cloud services offering a command-line utility to interact with their APIs). The container would have a local copy of the back-end utility, which would receive the batch when launched, process the batch, and return the results to the user via WebSocket. This implementation would require only small-scale modification of the current utility in order to successfully procedure batches, but would need to be combined with boosted features that can restrict apply of such cloud services.
four.5 Platform architecture
modelRxiv is browser-based, and was designed every bit a serverless application. This greatly reduces operational costs as there is no need to operate a dorsum-stop server, and costs calibration with utilize. To farther reduce running costs, dynamic content is served every bit static files: model code is uploaded as script files, while metadata is available as JavaScript Object Note (JSON) files. These are indexed as larger JSON files, reducing the need for database requests for common actions such as browsing the index of models or manipulating models. User authentication uses JSON Spider web Tokens (JWT), then that user information is retrieved only when logging in; subsequent requests are sent along with a token that contains the user data (specifically, the user ID). These considerations are important to make the projection sustainable in the long-term as a free, open up-source repository.
4.six Authentication
When registering to modelRxiv, users provide merely an arbitrary username ('Nickname') and password. Both the username and the password are stored encrypted and so that the modelRxiv platform and other users have no access to this information. A user ID is generated when registering, and when logging in, this user ID is encoded in a JWT using a private fundamental (RS256). When performing actions with the API (such equally uploading or editing models), the token is decoded using the public key, confirming that the JWT was encoded by the correct private key, and providing the user ID. To provide authenticated access to private static resources (such as sandboxed models), upon log-in in the user is provided with an additional token (a AWS CloudFront signed cookie) that provides access to a specific URL blueprint including the user ID. Both tokens take the same death time; when expired, users must log-in again to receive new tokens. Using JWTs and AWS CloudFront signed cookies reduces the need for authentication and database access. A like compages could be deployed to other deject providers.
5 Accessibility of data
The modelRxiv platform, including all models analyzed here, is freely accessible at https://modelrxiv.org. The platform lawmaking is available at github.com/carrowkeel/modelrxiv. All platform lawmaking is licensed under AGPLv3. Models are licensed under CC-BY 4.0 unless otherwise stated.
6 Author contributions
KDH designed and developed the platform, GH consulted on cloud integration and GG supervised the development of modeling features. KDH and GG wrote the manuscript. All authors read and approved the last version of the manuscript.
7 Funding
We would like thank David Gokhman, Oren Kolodny and Royi Zur for helpful comments and discussions. This project was supported by Israel Science Foundation (ISF) Grant 2049/21, and past German-Israel Foundation (GIF) Grant I-1526-500.15/2021.
References
- ane.↵
- two.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- viii.↵
- nine.↵
- ten.↵
- xi.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- eighteen.↵
Source: https://www.biorxiv.org/content/10.1101/2022.02.16.480599v1.full
Postar um comentário for "The Document Upload Process Has Encountered an Issue With Your Netbadge Credentials."