Automatically extract TML definitions from tml/export

ThoughtSpot elements such as search, Liveboards, and data connections are all defined in a JSON-based metadata definition called ThoughtSpot Modeling Language, or TML. Recently, I blogged about how you can use Postman to access platform APIs to import/export TML as part of your devops processes; for example, to check in TML definitions and push to another environment via a continuous integration process. 

The TML export is pretty straightforward. You create a POST request to /tml/exportand pass in an array of object ids for the ThoughtSpot elements to return in JSON.

curl -X POST
--header 'Accept: text/plain' \
--header 'X-Requested-By: ThoughtSpot' \
--data-urlencode 'export_ids=["12289fad-f230-485e-8c65-e36082eebf44"]' \
--data-urlencode 'formattype=YAML' \
--data-urlencode 'export_associated=false' \
'https://<ThoughtSpot-host>/callosum/v1/tspublic/v1/metadata/tml/export'

The resulting JSON file which is returned contains an array of nested TML definitions:

{

  "object":[

    {

      "info":{

        "name":"Basic Answer 1",

        "filename":"Basic Answer 1.answer.tml",

        "status":{

          "status_code":"OK"

        },

        "type":"answer",

        "id":"12289fad-f230-485e-8c65-e36082eebf44"

      },

      "edoc":"guid: 12289fad-f230-485e-8c65-e36082eebf44\n  .........\n"

    }

  ]

}

You can check in this response to source control, however, you can't import it directly into another environment using tml/import. Looking at the payload above, the specific TML we need to pass to a tml/import request is contained in the edoc attribute and added to the import_objects attribute (note: I abbreviated the example TML for readability. You can see a full example in the docs) 

curl -X POST --header 'Accept: text/plain' --header 'X-Requested-By: 
ThoughtSpot' --data-urlencode 'import_objects=[{
"guid": "12289fad-f230-485e-8c65-e36082eebf44",
..
}
}]'
--data-urlencode 'import_policy=PARTIAL'
--data-urlencode 'force_create=true' 
'https://<ThoughtSpot-host>/callosum/v1/tspublic/v1/metadata/tml/import'

Hello TMLify 👋

To make continuous integration easier, I wrote a little python script, TMLify, to automatically extract the TML from a tml/export JSON response and save it to a file. From here, you can pipe this into your tml/import import_objects as part of a devops script. You can grab tmlify here. Usage is pretty straightforward:

$ python tmlify.py outputoftmlexport.json outputfile.tml

Hopefully this helps save some time, and allows you to better integrate ThoughtSpot into your devops processes. TMLify is a nice simple standalone utility. If you need a complete solution, check out the TML tools project.