Appendix A JSON Data Structure Schema
This appendix describes the JSON schemas used in various SymetryML REST API HTTP request and response body. This is not a pure JSON schema definition because JSON schemas do not have a primitive type for hash map, while the SymetryML API uses hash maps extensively in request/response messages. The JSON type ‘object’ is the closest data structure to a hash map, as it can be seen as an infinite set of key=string and values=objects. When part of a JSON structure should be a hash map, the object type is used in the schema definition. When an entity contains an object - that is a hash map - additional comments are added that define the keys and values allowed. These comments are highlighted in red, so you can find them easily. The formats depicted in the following sections are important because this is how the REST server deserializes the HTTP request body and serializes the HTTP response body.
Request Body JSON
DataFrame JSON
DataFrame : {
"attributeTypes": {
"type": "array" {
"items": "string"
}
},
"attributeNames": {
"type": "array" {
"items": "string"
}
},
"data" : {
"type" : "array" {
"items" : "array" {
"items" : "string"
}
}
}
}MLContext JSON
MLContext : {
"targets" : "array" {
"items" : "string"
},
"inputAttributes" : "array" {
"items" : "string"
},,
"inputAttributeNames" : "array" {
"items" : "string"
},
"targetAttributeNames" : "array" {
"items" : "string"
}
"extraParameters" : object // key values hash map
}DSInfo JSON
DSInfo : {
"type" : "string",
"name" : "string",
"info" : "object",
"extra" : "object"
}TaskInfo JSON
TaskInfo : {
"type" : "string",
"name" : "string",
"params" : "object",
}DSListingRequest JSON
DSListingRequest : {
"type" : "string",
"path" : "string",
"info" : "object"
}ExploreContext JSON
The ExploreContext data structure is a list of MLContext. It allows to request for a given exploration metrics with multiple attributes or attributes pairs.
ExploreContext : {
"values" : "arrays" {
"items" : "MLContext"
}
}Request Response JSON
This is the main JSON response that all REST SymetryML calls return. Depending on the call, the values, if present, will be different.
JSONResponseClient : {
"statusCode ": {
"type" : "number",
"description" : "HTTP/HTTPS response code",
"required" : true,
"default" : "200"
},
"statusString" : {
"type" : "String",
"description" : "Contextual error information if needed",
"required" : true,
"default" : "No Error"
},
"values" : {
"type" : "object" {
"properties" : { // the different type of entities
"KSKVMap" : {
"type" : "KSVSMap"
},
"modelInfoResponse" : {
"type" : "ModelInfo"
},
"status" : {
"type" : "string"
},
"stringToSign" :{
"type" : "string"
},
"KSVSMap" : {
"type" : "KSVDMap"
},
"smlInfo" : {
"type" : "ProjectInfo"
},
"smlInfoList" : {
"type" : "ProjectInfoList"
},
"stringList" : {
"type" : "StringList"
},
"jobinfo" : {
"type" : "JobInfo"
},
"smlOptions" : {
"type" : "SymetryProjectOptions"
},
"dataframe" : {
"type" : "DataFrame"
},
"dsinfo" : {
"type" : "DSInfo"
},
"dsdirectoryListing" : {
"type" : "DSListingResponse"
},
"pcaValues" : {
"type" : "DataFrame"
},
"pcaVectors" : {
"type" : "DataFrame"
},
"pcaSumValues" : "number"
},
"federationInfo" : {
"type" : "FederationInfo"
},
"cellsStatus" : {
"type" : "cellsStatusEncrypted"
},
"fedSyncStats" : {
"type" : "FedSyncStats"
}
}
}
}Response Entities
Response entities are JSON data structures that are returned given a specific REST request. This secondary response is added to the main response values field.
DSListingResponse JSON
DSListingResponse : {
"ok" : "Boolean",
"files" : {
"type" : "array" {
"items" : "string"
}
},
"dirs" : {
"type" : "array" {
"items" : "string"
}
},
"path" : "string",
"marker" : "string",
"delimiter" : "string"
}JobInfo
JobInfo : {
"jobId" : "number",
"jobType" : "string",
"owner" : "string",
"startingDate" : "number",
"current" : "number",
"maximum" : "number",
"isOnBoundedExecutor" : "Boolean",
"isStarted" : "Boolean",
"isDone" : "Boolean",
"details" : "object" // <string, string> map
}KSVDMap
KSVDMap : {
"values" : {
"type" : "array" {
"items" : "object" {
// key : string
// value : number
}
}
}FedSyncStats
FedSyncStats : {
"sndCount" : "number", // # of time this peer tried to sync
"sndPSRContratError" : "number", // # of time this peer tried to sync with error
"rcvCount" : "number", // # of time we received sync from other peers
"rcvError" : "number", // # of time we received sync from other peers with error
"rcvPSRContractError" : "object", // keep count of # error by peer id
"errorLog" : "array" {
"items" : "String" // list of sync error
}
"syncLog" : "array" {
"items" : "String" // list of sucessful sync message
}
}SymetryProjectOptions
SymetryProjectOptions : {
"gpuEnabled" : "number"
}KSVSMap
Prediction services use KSVSMap, which consists of a list of map<string, string>.
KSVSMap : {
"values" : {
"type" : "array" {
"items" : "object" {
// key : string
// value : string
}
}
}ModelInfo
ModelInfo : {
"modelId" : "string",
"modelType" : "string",
"modelBuilt" : "number",
"buildTime" : "number",
"attributeNames" : {
"type" : "array" {
"items" : "string"
}
},
"attributeIndexes" : {
"type" : "array" {
"items" : "number"
}
},
"attributeTypes" : {
"type" : "array" {
"items" : "string"
}
},
"predictInfo" : { // value returned by the predictions.
"type" : "array" {
"items" : "string"
}
},
"scoreInfo" : { // value containing score in predictions.
"type" : "array" {
"items" : "string"
}
},
"targetNames" : {
"type" : "array" {
"items" : "string"
}
},
"targetindexes" : {
"type" : "array" {
"items" : "number"
}
},
"info" : { // additional information, model type dependent.
"type" : "KSVSMap"
},
"creationDate" : "number"
"extra": "Object" // For MLR and Kaplan Meier models contains additional info. The 'object' is effectively a map where keys are strings and value are Dataframe see "DataFrame JSON" section earlier.
}FusionCellStatus
Fusion Status information
FusionCellStatus : {
"host" : "string", // host
"cid" : "string", // customer id
"status" : "string", // status
"pid" : "string", // project id
}FusionCellStatusList
List of Fusion Status information
FusionCellStatusList : {
"values" : {
"type" : "array" {
"items" : "FusionCellStatus"
}
}
}FusionCellInfo
Fusion Cell information
FusionCellInfo : {
"host" : "string", // host
"cid" : "string", // customer id
"sk" : "string", // secret key
"pid" : "string", // project id
}FusionCellInfoList
Fusion Cell information
FusionCellInfoList : {
"values" : {
"type" : "array" {
"items" : "FusionCellInfo"
}
}
}ProjectInfo
The SymetryML project info request uses ProjectInfo.
ProjectInfo : {
"pid" : "string",
"encoderName" : "string",
"modelsList" : {
"type" : "array" {
"items" : "string"
},
"modelTypeList" : {
"type" : "array" {
"items" : "string"
}
},
"dsList" : {
"type" : "array" {
"items" : "string"
}
},
"attributeNames" : {
"type" : "array" {
"items" : "string"
}
},
"attributeIndexes" : {
"type" : "array" {
"items" : "number"
}
},
"attributeTypes" : { // [c,b,x,s,l,-]
"type" : "array" {
"items" : "string"
}
},
"modelAssessments" : {
"type" : "array" {
"items" : "string"
}
},
"modelPredictions" : {
"type" : "array" {
"items" : "string"
}
},
"partitionColumn" : "string",
"params" : "object", // key values hash map <string, string>
"hash" : "number", // internal use
"categorySeparator" : "string",
"type" : "string", //
"creationDate" : "number",
"lastModificationDate" : "number",
"loaded" : "true|false",
"persisted" : "true|false",
"histogramEnabled" : "true|false",
"autoSave" : "true|false",
"fusionCellInfo" : "string" // encrypted json of fusioncellinfolist
}StringList
StringList can be returned by various REST calls.
StringList : {
"type" : "array" {
"items" : "string"
}
}ProjectInfoList
ProjectInfoList is returned after restoring a specific SymetryML project or restoring all SymetryML projects for a specific customer.
ProjectInfoList : {
"values" : {
"type" : "array" {
"items" : "ProjectInfo"
}
}
}FederationInfo
The SymetryML Federation info request uses FederationInfo.
FederationInfo : {
"aws_id" : "string",
"aws_access_key" : "string",
"aws_queue_url" : "string",
"aws_secret_key" : "string",
"aws_topic_arn" : "string",
"fed_local_id" : "string",
"fed_name" : "string",
"fed_secret_key" : "string",
"sync_sched" : "string",
"fed_type" : "string",
"local_uuid" : "string",
"s3_bucket" : "string",
"is_pulsing" : "boolean",
"peersInfo" : "object", // key values hash map<string, string>
"psr_contract" : "string", // the psr contract for this federation, if it exists
"rcv_error_action" : "string", // psr contract receive error action
"snd_error_action" : "string", // psr contract send error action
"fed_use_smpc" : "string", // whether this federation use SMPC or not
"allow_peers_explore" : boolean // whether this peer allows peer exploration or not
}AwsInfo
Amazon Web Service - AWS - information. This information will be returned as an encrypted JSON string.
public class AwsInfo {
/**
* map<AWS_REGION, List<Map<topic_arn, List<Map<topic_arn_key, topic_arn_value>>>>>
*/
public Map<String, List<Map<String, List<Map<String, String>>>>> topics;
/**
* Map<AWS_REGION, List<sqs_queue_name>>
*/
public Map<String, List<String>> queues;
// Example:
/*
{
"topics":{"US_WEST_2":
[{"arn:aws:sns:us-west-2:123456789012:fed1":
[{"owner":"123456789012",
"protocol":"sqs",
"endpoint":"arn:aws:sqs:us-west-2:123456789012:fed1_proj1",
"subscription_arn":"arn:aws:sns:us-west-2:123456789012:fed1:abcdefg1-8616-4c57-873f-abcdfef12345"}
]}
],
"US_EAST_1":
[{"arn:aws:sns:us-east-1:123456789012:FEEED":
[{"owner":"123456789012",
"protocol":"sqs",
"endpoint":"arn:aws:sqs:us-east-1:123456789012:FEEED_toto",
"subscription_arn":"arn:aws:sns:us-east-1:123456789012:FEEED:abcdefg1-8eaf-4c22-8062-abcdfef12345"}]
},
{"arn:aws:sns:us-east-1:123456789012:f200":
[{"owner":"123456789012",
"protocol":"sqs",
"endpoint":"arn:aws:sqs:us-east-1:123456789012:f200_p1",
"subscription_arn":"arn:aws:sns:us-east-1:123456789012:f200:abcdefg1-a665-4e82-961b-abcdfef12345"}]
}
},
"queues":{
"US_WEST_2":[
"https://sqs.us-west-2.amazonaws.com/123456789012/fed1_proj1"],
"US_EAST_1":[
"https://sqs.us-east-1.amazonaws.com/123456789012/FEEED_toto",
"https://sqs.us-east-1.amazonaws.com/123456789012/f200_p1",
]
}
}
*/
}Density
DensityList contains a list of Density
Density : {
"attrName" : "string",
"min" : "number",
"max" : "number",
"width" : "number",
"histogram" : "array" {
"items" : "number"
},
"error" : "string"
}Density List
DensityList contains a list of Density objects.
DensityList : {
"values" : {
"type" : "array" {
"items" : "Density"
}
}
}Last updated