Command Line Interface API Reference¶
neurocaas-contrib¶
Base command to interact with the neurocaas_contrib repo from the command line. Can be given location and analysis name parameters to run certain actions in a one-off manner, but preferred workflow is assigning these with the init subcommand. Assigns parameters (analysis metadata, blueprint, active image) to be referenced in other subparameters.
neurocaas-contrib [OPTIONS] COMMAND [ARGS]...
Options
- --location <location>¶
Root location for neurocaas local environment build.
- -a, --analysis-name <analysis_name>¶
Name of the analysis you want to build.
neurocaas-contrib remote¶
develop remotely on an AWS instance.
neurocaas-contrib remote [OPTIONS] COMMAND [ARGS]...
assign-instance¶
Assign a new instance to a NeuroCAASAMI object
neurocaas-contrib remote assign-instance [OPTIONS]
Options
- -i, --instance <instance>¶
instance id to assign to this instance.
- -n, --name <name>¶
name for this assigned instance
- -d, --description <description>¶
description of this instance.
create-devami¶
save the current development instance into an ami.
neurocaas-contrib remote create-devami [OPTIONS]
Options
- -n, --name <name>¶
name to give to the new ami
develop-remote¶
Initialize a NeuroCAASAMI object
neurocaas-contrib remote develop-remote [OPTIONS]
Options
- -i, --index <index>¶
if there are multiple development histories saved, index into them
end-session¶
Finish development session.
neurocaas-contrib remote end-session [OPTIONS]
Options
- -f, --force¶
if true, will delete new instances even if they haven’t been saved to amis
extend-lifetime¶
Get the lifetime of the instance
neurocaas-contrib remote extend-lifetime [OPTIONS]
Options
- -m, --minutes <minutes>¶
number of minutes to extend lifetime by.
job-output¶
get the output from the most recently run job.
neurocaas-contrib remote job-output [OPTIONS]
Options
- -j, --jobind <jobind>¶
index of job to get the output for
job-status¶
get the status from the most recently run job.
neurocaas-contrib remote job-status [OPTIONS]
Options
- -j, --jobind <jobind>¶
index of job to get the output for
launch-devinstance¶
Launch a new instance from an ami
neurocaas-contrib remote launch-devinstance [OPTIONS]
Options
- -a, --amiid <amiid>¶
AMI id.
- -n, --name <name>¶
Name of instance
- -d, --description <description>¶
Description for instance
- -v, --volumesize <volumesize>¶
size of the volume you want to attach to this instance.
- -t, --timeout <timeout>¶
length of timeout to associate with this instance
select-instance¶
Select instance from pool.
neurocaas-contrib remote select-instance [OPTIONS]
Options
- -i, --instance_id <instance_id>¶
Instance ID. Must give ID or name.
- -n, --name <name>¶
Instance name. Must give id or name.
start-devinstance¶
Start the current instance
neurocaas-contrib remote start-devinstance [OPTIONS]
Options
- -t, --timeout <timeout>¶
length of timeout to associate with this instance
start-session¶
Start developing remotely.
neurocaas-contrib remote start-session [OPTIONS]
Options
- -f, --force¶
if true, will delete new instances even if they haven’t been saved to amis
submit-job¶
submit a job to your instance. Optionally, you can upload datasets and config files to s3 if you will be referencing them in your submitpath for a faster turnaround.
neurocaas-contrib remote submit-job [OPTIONS]
Options
- -s, --submitpath <submitpath>¶
path to submitfile
- -d, --data <data>¶
path to test dataset.
- -c, --config <config>¶
path to test config file.
terminate-devinstance¶
Terminate the current development instance
neurocaas-contrib remote terminate-devinstance [OPTIONS]
Options
- -f, --force <force>¶
whether or not to force deletion.
update-blueprint¶
update the blueprint with most recently developed amis.
neurocaas-contrib remote update-blueprint [OPTIONS]
Options
- -a, --amiid <amiid>¶
id of new ami (will default to newest in list if not given)
- -m, --message <message>¶
message associated with the commit carried out with this command.
neurocaas-contrib workflow¶
tools to manage the transfer of data to and from this development
neurocaas-contrib workflow [OPTIONS] COMMAND [ARGS]...
cleanup¶
takes care of post processing: sends end and config files to the bucket.
neurocaas-contrib workflow cleanup [OPTIONS]
get-bucket¶
get bucket name registered if registered.
neurocaas-contrib workflow get-bucket [OPTIONS]
get-config¶
get a registered config from S3
neurocaas-contrib workflow get-config [OPTIONS]
Options
- -o, --outputpath <outputpath>¶
path to write output to.
- -f, --force¶
if true, will redownload even if exists at intended output location
- -d, --display¶
if true, will show download progress
get-configname¶
get the name of the config file you registered
neurocaas-contrib workflow get-configname [OPTIONS]
get-configpath¶
get the path to the config file you registered
neurocaas-contrib workflow get-configpath [OPTIONS]
get-data¶
get a registered dataset from S3
neurocaas-contrib workflow get-data [OPTIONS]
Options
- -o, --outputpath <outputpath>¶
path to write output to.
- -f, --force¶
if true, will redownload even if exists at intended output location
- -d, --display¶
if true, will show download progress
get-dataname¶
get the name of the dataset you registered
neurocaas-contrib workflow get-dataname [OPTIONS]
get-datapath¶
get the path to the dataset you registered
neurocaas-contrib workflow get-datapath [OPTIONS]
get-file¶
get a registered file from S3
neurocaas-contrib workflow get-file [OPTIONS]
Options
- -n, --name <name>¶
name used to register this file.
- -o, --outputpath <outputpath>¶
path to write output to.
- -f, --force¶
if true, will redownload even if exists at intended output location
- -d, --display¶
if true, will show download progress
get-filename¶
get the name of the file you registered
neurocaas-contrib workflow get-filename [OPTIONS]
Options
- -n, --name <name>¶
name of the file you registered.
get-filepath¶
get the path to the file you registered
neurocaas-contrib workflow get-filepath [OPTIONS]
Options
- -n, --name <name>¶
name of the file you registered.
get-group¶
get the name of the group whose data you are reading in.
neurocaas-contrib workflow get-group [OPTIONS]
get-resultpath¶
get the s3 path a local file or directory would be uploaded to
neurocaas-contrib workflow get-resultpath [OPTIONS]
Options
- -l, --locpath <locpath>¶
path to file you want to get remote path for.
get-resultpath-tmp¶
get the path to a temporary result location
neurocaas-contrib workflow get-resultpath-tmp [OPTIONS]
initialize-job¶
initialize data transfer location
neurocaas-contrib workflow initialize-job [OPTIONS]
Options
- -p, --path <path>¶
location where we will store and register data.
log-command¶
runs a script with commands, all specified as a string
neurocaas-contrib workflow log-command [OPTIONS]
Options
- -c, --command <command>¶
the script (with arguments) you want to run.
- -b, --bucket <bucket>¶
bucket where results are located
- -r, --resultfolder <resultfolder>¶
path within the bucket to the results folder where we expect logs.
log-command-local¶
runs a script with commands, but only locally.
neurocaas-contrib workflow log-command-local [OPTIONS]
Options
- -c, --command <command>¶
the script (with arguments) you want to run.
put-result¶
put a file into the result directory in s3
neurocaas-contrib workflow put-result [OPTIONS]
Options
- -r, --resultpath <resultpath>¶
local location of result file
- -d, --display¶
if true, will show download progress
register-config¶
register a config file for processing. Can be located locally (use -l) or in s3 (use -b, -k)
neurocaas-contrib workflow register-config [OPTIONS]
Options
- -b, --bucket <bucket>¶
bucket where config file is located
- -k, --key <key>¶
key of the file within the indicated bucket
- -l, --localpath <localpath>¶
local path where config is located.
register-dataset¶
register a dataset for processing. Can be located locally (use -l) or in s3 (use -b, -k)
neurocaas-contrib workflow register-dataset [OPTIONS]
Options
- -b, --bucket <bucket>¶
bucket where data is located
- -k, --key <key>¶
key of the file within the indicated bucket
- -l, --localpath <localpath>¶
local path where data is located.
register-file¶
register an arbitrary file for processing. can be located locally (use -l) or in s3 (use -b, -k)
neurocaas-contrib workflow register-file [OPTIONS]
Options
- -n, --name <name>¶
name of file to reference later
- -b, --bucket <bucket>¶
bucket where file is located
- -k, --key <key>¶
key of the file within the indicated bucket
- -l, --localpath <localpath>¶
local path where file is located.
register-resultpath¶
register a result path to store results. can be located locally (use -l) or in s3 (use -b, -k)
neurocaas-contrib workflow register-resultpath [OPTIONS]
Options
- -b, --bucket <bucket>¶
bucket where results should be located
- -k, --key <key>¶
subkey of the folder where results are stored within the indicated bucket. The data will be put at s3://bucket/{groupname}/key, where groupname comes from the registered dataset.
- -l, --localpath <localpath>¶
local path where config is located.
neurocaas-contrib scripting¶
Scripting functions.
neurocaas-contrib scripting [OPTIONS] COMMAND [ARGS]...
parse-zip¶
extract zipped folder into the same directory, and echo basename of the folder that is extracted.
neurocaas-contrib scripting parse-zip [OPTIONS]
Options
- -z, --zippath <zippath>¶
path to zip file
- -o, --outpath <outpath>¶
directory in which to place the extracted directory- default to same directory.
read-yaml¶
extract field from a yaml file as a string. If field is a list, will output into a bash array.
neurocaas-contrib scripting read-yaml [OPTIONS]
Options
- -d, --default <default>¶
default output to give if not found
- -f, --field <field>¶
field to extract from yaml file
- -p, --path <path>¶
path to yaml file
neurocaas-contrib monitor¶
Job monitoring functions.
neurocaas-contrib monitor [OPTIONS] COMMAND [ARGS]...
describe-certificate¶
print certificate file for submission. can give submitpath or groupname and timestamp.
neurocaas-contrib monitor describe-certificate [OPTIONS]
Options
- -s, --stackname <stackname>¶
name of the stack folder that you want to get job manager requests for.
- -p, --submitpath <submitpath>¶
path to submit file
- -g, --groupname <groupname>¶
name of the group we are getting certificate for
- -t, --timestamp <timestamp>¶
timestamp of job.
describe-datasets¶
print datasets being analyzed, and instances on which they are being analyzed.
neurocaas-contrib monitor describe-datasets [OPTIONS]
Options
- -s, --stackname <stackname>¶
name of the stack folder that you want to get job manager requests for.
- -p, --submitpath <submitpath>¶
path to submit file
describe-datastatus¶
prints the datastatus of any given dataseet.
neurocaas-contrib monitor describe-datastatus [OPTIONS]
Options
- -s, --stackname <stackname>¶
name of the stack folder that you want to get job manager requests for.
- -p, --submitpath <submitpath>¶
path to submit file
- -g, --groupname <groupname>¶
name of the group we are getting certificate for
- -t, --timestamp <timestamp>¶
timestamp of job.
- -d, --dataname <dataname>¶
basename of dataset to get status for.
- -c, --cutoff <cutoff>¶
cutoff for logs (c-end)
describe-job-manager-request¶
print recent job manager requests.
neurocaas-contrib monitor describe-job-manager-request [OPTIONS]
Options
- -s, --stackname <stackname>¶
name of the stack folder that you want to get job manager requests for.
- -h, --hours <hours>¶
how many hours ago you want to start looking
- -i, --index <index>¶
the index of request you want to get (0 = most recent)
poll-analysis¶
poll an ongoing analysis for results.
neurocaas-contrib monitor poll-analysis [OPTIONS]
Options
- -s, --stackname <stackname>¶
name of the stack folder that you want to get job manager requests for.
- -p, --pathprefix <pathprefix>¶
path to S3
- -l, --localpath <localpath>¶
local file path where you want to dump these results and logs.
- -i, --interval <interval>¶
interval (in seconds) between polls (default 1 min)
- -t, --timeout <timeout>¶
timeout in seconds at which to quit polling (default 15 mins)
neurocaas-contrib local¶
develop locally in a docker container.
neurocaas-contrib local [OPTIONS] COMMAND [ARGS]...
enter-container¶
enter the container to start development.
neurocaas-contrib local enter-container [OPTIONS]
Options
- -c, --container <container>¶
name of the container to enter.
get-iae-info¶
print information about the IAE from the blueprint.
neurocaas-contrib local get-iae-info [OPTIONS]
reset-container¶
removes the current active container.
neurocaas-contrib local reset-container [OPTIONS]
Options
- -c, --container <container>¶
name of container to remove
run-analysis¶
run analysis in a saved environment.
neurocaas-contrib local run-analysis [OPTIONS]
Options
- -c, --config <config>¶
name of the config file you will analyze.
- -d, --data <data>¶
name of the dataset you will analyze.
- -i, --image <image>¶
image to run analysis from.
save-developed-image¶
save a container to a new image.
neurocaas-contrib local save-developed-image [OPTIONS]
Options
- -t, --tagid <tagid>¶
unique tag identifier to identify this image. Will generate image name as neurocaas/contrib:[analysisname].[tagid]
- -f, --force¶
if true, overwrites existing image with this name.
- -c, --container <container>¶
name of the container to save.
- -s, --script <script>¶
path to script inside the container.
setup-development-container¶
set up an environment to develop in.
neurocaas-contrib local setup-development-container [OPTIONS]
Options
- -c, --container <container>¶
name to give to containers
- -i, --image <image>¶
base image tag to use as the start of development. If not given, reverts to entry in blueprint.