Admin Built-in Commands
The following commands must be ran with challengeutils
, e.g.
$ challengeutils <built-in command> <options>
Note that the majority of the commands listed here will require certain
permissions on the relevant Sypanse entities, e.g. admin
permissions
or score
permissions.
For more information on Synapse permissions, see here.
Create a challenge
Synopsis
- create-challenge
“CHALLENGE NAME HERE” [–livesiteid SYNID]
Description
Create a challenge space in Synapse, including two Projects and four Teams that are needed for a challenge. For more information on various Challenge components, see challenge administration.
Mirror wiki changes
Synopsis
- mirror-wiki
source_id dest_id [–force]
Description
Sync changes made from a source wiki to the destination wiki, e.g. from staging to live. Updates will only occur in wiki pages that have matching titles in the source wiki.
Important
This command is different from copying wikis. To copy, see synapseutils.
For best practice, make all edits to the staging site of a challenge only, then use this command to sync over the changes to the live site.
List evaluations
Synopsis
- list-evaluations
project_id
Description
List all evaluation queues of a Synapse project.
Set an evaluation quota
Synopsis
- set-evaluation-quota
eval_id [–round_start yyyy-MM-ddTHH:mm:ss [–round_end yyyy-MM-ddTHH:mm:ss] [–round_duration n] [–num_rounds n] [–sub_limit n]
Description
Set the quota of an evaluation queue. Quota options include the round starting date, round ending date, round duration, number of rounds, and submission limit.
Warning
When this command is used, all settings previously set for the queue will be erased. For any settings you do not want to update or remove, pass the original values into the optional parameters defined below.
Optional
-
--round_start
Start of round in yyyy-MM-ddTHH:mm:ss (local military time) format
-
--round_end
End of round in yyyy-MM-ddTHH:mm:ss (local military time) format; do not use with
round_duration
-
--round_duration
Round duration in milliseconds; do not use with
round_end
-
--num_rounds
Number of rounds (must set for time-related quotas to work)
-
--sub_limit
Number of submissions allowed per round
Update an evaluation ACL
Synopsis
- set-evaluation-acl
evalid user_or_team permission_level
Description
Set the evaluation permissions for user_or_team
with
permission_level
access.
Query an evaluation
Synopsis
- query
“QUERY” [–outputfile file] [–render] [–limit n] [–offset n]
Description
Query an evaluation queue.
Download a submission
Synopsis
- download-submission
sub_id [–download_location path] [–output file]
Description
Download a Submission object.
Annotate a submission
Synopsis
- annotate-submission
sub_id json_file [-p ] [-f]
Description
Annotate a Submission object with a JSON file. The file should comprise of a list of key-value pairs, where the key is the annotation and the value is the annotation value, e.g.
{
"round": 1,
"score": 100
}
Update a submission status
Synopsis
- change-status
sub_id status
Description
Update the status
annotation of a Submission Status object.
Stop a Docker submission
Synopsis
- stop-submission-over-quota
eval_id time_quota
Description
Terminate a Docker submission (usually applies to submissions that have a runtime longer than the alloted time).
Update an entity ACL
Synopsis
- set-entity-acl
ent_id user_or_team permission_level
Description
Set the entity permissions for user_or_team
with
permission_level
access.
Validate a Project Submission
Synopsis
- validate-project
submissionid challengewiki permission_level [-p] [-a] [-o]
Description
Validates a Synapse Project submission by making sure it is shared with the correct users.
Archive a Project Submission
Synopsis
- archive-project
submissionid admin [-o]
Description
Archives a Project submission by copying it
Validate a Docker Submission
Synopsis
- validate-docker
sub_id [-o]
Description
Validates a Docker Submission to make sure the image and sha-digest exists.
Pull a Synapse Project’s Wiki
Synopsis
- pull-wiki
projectid [–workdir]
Description
Download each wiki page of a synapse project as individual .md files and saves a json configuration (wiki_config.json).
Push a Synapse Project’s Wiki
Synopsis
- push-wiki
projectid [–workdir]
Description
Updates Synapse wiki by uploading markdown files linked together by a wiki_config.json.
The markdown and wiki_config.json can be obtained by the pull-wiki
command. The
wiki_config.json is validated prior to any updates, if there are any valiation errors,
your wiki pages will not be updated.
Scan an evaluation queue for cheating
Synopsis
- cheat-detection
evaluationid [-s]
Description
Scans an evaluation queue for possible pairs of users who are trying to skirt the daily submission limit by creating multiple users to submit to a challenge. A report is generated with possible linked users and the strength of evidence that any two users are linked.