Automatically extract TML definitions from tml/export

ThoughtSpot elements such as search, Liveboards, and data connections are all defined in a JSON-based metadata definition called ThoughtSpot Modeling Language, or TML. Recently, I blogged about how you can use Postman to access platform APIs to import/export TML as part of your devops processes; for example, to check in TML definitions and push to another environment via a continuous integration process. 

The TML export is pretty straightforward. You create a POST request to /tml/exportand pass in an array of object ids for the ThoughtSpot elements to return in JSON.

curl -X POST<br>--header 'Accept: text/plain' \<br>--header 'X-Requested-By: ThoughtSpot' \<br>--data-urlencode 'export_ids=["12289fad-f230-485e-8c65-e36082eebf44"]' \<br>--data-urlencode 'formattype=YAML' \<br>--data-urlencode 'export_associated=false' \<br>'http://<ThoughtSpot-host>/callosum/v1/tspublic/v1/metadata/tml/export'

The resulting JSON file which is returned contains an array of nested TML definitions:

{<br>  "object": [<br>    {<br>      "info": {<br>        "name": "Basic Answer 1",<br>        "filename": "Basic Answer 1.answer.tml",<br>        "status": {<br>          "status_code": "OK"<br>        },<br>        "type": "answer",<br>        "id": "12289fad-f230-485e-8c65-e36082eebf44"<br>      },<br>      "edoc": "guid: 12289fad-f230-485e-8c65-e36082eebf44\n  .........\n"<br>    }<br>  ]<br>}

You can check in this response to source control, however, you can't import it directly into another environment using tml/import. Looking at the payload above, the specific TML we need to pass to a tml/import request is contained in the edoc attribute and added to the import_objects attribute (note: I abbreviated the example TML for readability. You can see a full example in the docs

curl -X POST --header 'Accept: text/plain' --header 'X-Requested-By:

ThoughtSpot' --data-urlencode 'import_objects=[{<br>"guid": "12289fad-f230-485e-8c65-e36082eebf44",<br>..<br>}<br>}]'<br>--data-urlencode 'import_policy=PARTIAL'<br>--data-urlencode 'force_create=true'

'http://<ThoughtSpot-host>/callosum/v1/tspublic/v1/metadata/tml/import'

Hello TMLify 👋

To make continuous integration easier, I wrote a little python script, TMLify, to automatically extract the TML from a tml/export JSON response and save it to a file. From here, you can pipe this into your tml/import import_objects as part of a devops script. You can grab tmlify here. Usage is pretty straightforward:

$ python tmlify.py outputoftmlexport.json outputfile.tml

Hopefully this helps save some time, and allows you to better integrate ThoughtSpot into your devops processes. TMLify is a nice simple standalone utility. If you need a complete solution, check out the TML tools project