What Are Sim Tasks?
Sim Tasks are a business object in Openlink’s system that handles end-of-day and ad-hoc simulations. Openlink created sim tasks in version 11 as an enhancement to the Batch Sim Definition framework.
Batch sim definitions are the typical way that users would configure a run of portfolio simulations to calculate valuations, Greeks, VaR, and other analytical jobs. The batch sim definition screen is serviceable, but there is no API that allows for the easy creation and update of the definition. There are also hard-coded relationships implied by some of the properties of the batch sim, such as the reval type to use for the retrieval of prior day simulation results.
Why Should I Use Sim Tasks?
We first started to use Sim Tasks extensively when a client’s use case called for the creation of new portfolios intraday. The client has hundreds of portfolios that contain sometimes only a handful of trades to track the trades’ P&L by trading strategy.
That client’s requirement was impossible to accommodate without the existence of an Openlink API to update a batch sim definition. The only way to amend a batch sim definition is via the GUI or to run a stored procedure. The GUI is too laborious, and running a stored procedure is an operation that is explicitly outside the bounds of Openlink’s support agreement.
Sim tasks are easy to create on-the-fly via API for a newly created portfolio.
A second client requirement that arose necessitated the use of Trade Snapshots. We published a paper on trade snapshots here and the OpenComponents and OpenJVS APIs here. The most common use-case for using trade snapshots is for an OTC derivative that was later terminated. Other use cases may include partial terminations, novations, amendments to the terms and conditions (where you want to retain the original, perhaps incorrect terms), and portfolio moves.
Batch sim definitions cannot run on a portfolio of trade snapshots. Sim tasks can run a trade snapshot, which means that it is possible to revalue a trade for a prior period that has been subsequently amended.
How to Configure a Sim Task
Sim Tasks can be created, updated and executed from the desktop using the Data Explorer view.
A batch sim definition is used to control the following fields:
- Query: filters the trades, typically isolating the portfolio
- Simulation Definition: the analytical results to run, and the configuration of any modifications
- Reval Type: EOD, Accounting, VaR, or another.
- All of these fields can be configured on a Sim Task, but there are many other fields that are also supported.
We already mentioned sim task support for ‘as-of’ trade snapshots. There is also support available to control the market data (closing, universal, or a personal market data set), the prior reval type, and the target run site. The run site configuration allows for precise control at runtime for where to run the sim task, enabling more easy control of job distribution across a cluster.
Openlink created an API to easily create and update sim task definitions. There is also an API to run a sim task, which generates its results and will save them to the database.
Sim tasks are business objects in the system, which means that it is possible to query for them in the Data Explorer. Users will typically setup a saved query that returns all sim tasks that have the EOD Reval Type.
The Downside to Sim Tasks: No Standard Content
Openlink has a standard plugin for running a batch sim as part of the end-of-day workflow. It is a simple plugin, and it is easy to change if you use a batch sim definition that differs from the standard name ‘EOD All Portfolios’.
There is no equivalent standard, Openlink-supported, plugin to run a collection of sim tasks. The API, however, is simple and a sample is included below. The sample is easily extended if you want to run sim tasks multiple times per day, or in batches.
There is also no standard plugin supported by Openlink to create a new sim task. We have also included a sample plugin below that creates a new sim task.
While Openlink does not deliver standard plugins to support these features, they will support any bugs found in the API. We have only ever encountered one sim task bug, and it was resolved in 2015. While it is a relatively recent feature, it was made generally available with a high level of quality.
Sample Plugins
This section includes sample plugins to create sim task definitions and to execute sim task jobs. These samples use Openlink’s OpenComponents SDK. Similar features are available using OpenJVS, and we can help with that too.
If you have any questions, contact us.
Sample End-of-Day Sim Task Execution Job
This is a snippet from one client’s production job. They have additional features to track execution time and memory consumption, not shown here. We can help add those features if you would like to know how.
package com.findur.oc.eod;
import java.util.Date;
import com.findur.oc.util.*;
import com.olf.embedded.*;
import com.olf.openrisk.*;
@ScriptCategory({EnumScriptCategory.Generic})
public class ExecuteSimTasks extends AbstractGenericScript {
/**
* The name of the Sim Task query
*/
private static String SIM_TASK_QUERY_NAME = "EOD Batch Sim";
private int tradeSnapshotTypeId = -1;
private Session session;
private MessageLog messageLog;
public ExecuteSimTasks() {
}
public Table execute(final Session session, final ConstTable table) {
this.session = session;
messageLog = new MessageLog(session, this.getClass());
tradeSnapshotTypeId = session.getStaticDataFactory().getReferenceObject(EnumReferenceObject.TradeSnapshotType, "EOD").getId();
int[] batchSimScope = getBatchSimScope();
if (batchSimScope != null) {
runBatchSim(batchSimScope);
}
return null;
}
/**
* Gets the name of the sim task query to run. This controls which sim tasks
* will be executed.
*/
public String getSimTaskQueryName() {
return SIM_TASK_QUERY_NAME;
}
/**
* Retrieves the Sim Task ids to run
*/
private int[] getBatchSimScope() {
String queryName = getSimTaskQueryName();
return QueryHelper.getObjectIds(session, queryName);
}
/**
* Executes each of the sim tasks in scope for the query
*/
private void runBatchSim(int[] simTaskIds) {
int counter = 0;
for (int simTaskId : simTaskIds) {
messageLog.info("Running sim task [" + (counter + 1) + "] of [" + simTaskIds.length + "]");
Task simTask = null;
SimResults simResults = null;
try {
simTask = session.getSimulationFactory().retrieveTask(simTaskId);
if (portfolioHasTrades(simTask) || portfolioHasSavedSnapshot(simTask)) {
simResults = simTask.execute();
} else {
messageLog.info("[" + simTask.getName() + "] has no trades. Skipping task execution");
}
} catch (Exception e) {
messageLog.error("Failed to execute sim task [" + simTask.getName() + "], ID [" + simTaskId + "]");
messageLog.error(e.getMessage());
//Decision: some clients do not want to throw an exception
//They will run a report to see if all required results are available
} finally {
dispose(simResults);
}
counter++;
}
}
/**
* Checks if there are trades associated with the sim task
*/
private boolean portfolioHasTrades(Task simTask) {
String queryName;
try {
String tradeSource = simTask.getValueAsString(simTask.getFieldId("Trade Source"));
if(tradeSource.equalsIgnoreCase("Query")) {
queryName = simTask.getValueAsString(simTask.getFieldId("Query"));
}
else {
queryName = simTask.getValueAsString(simTask.getFieldId("Trade Snapshot Name"));
}
int numTrades = QueryHelper.getObjectIds(session, queryName).length;
if (numTrades > 0) {
return true;
}
return false;
} catch (Exception e) {
messageLog.error("Error running query associated with sim task [" + simTask.getName() + "]");
messageLog.error(e.getClass() + ": " + e.getMessage());
//By default, be conservative and try to run the sim task
return true;
}
}
/**
* Checks if there is a saved snapshot associated with the sim task
*/
private boolean portfolioHasSavedSnapshot(Task simTask) {
String tradeSource = simTask.getValueAsString(simTask.getFieldId("Trade Source"));
if(tradeSource.equalsIgnoreCase("Query")) {
return false;
} else {
String snapshotName = simTask.getValueAsString(simTask.getFieldId("Trade Snapshot Name"));
String sql = "SELECT "
+ "\nFROM trade_snapshot_header tsh "
+ "\nWHERE tsh.snapshot_date = '" + session.getCalendarFactory().getSQLString(PluginHelper.getCurrentDate(session)) + "' "
+ "\nAND tsh.snapshot_ref_name = '" + snapshotName + "' "
+ "\nAND snapshot_type = " + tradeSnapshotTypeId;
Table table = null;
try {
table = session.getIOFactory().runSQL(sql);
if(table.getRowCount() > 0) {
return true;
}
} finally {
dispose(table);
}
}
return false;
}
}Sample Sim Task Creation Job
This plugin creates a sim task for a newly created portfolio. There are parameters in the plugin to control the name of the sim task, whether to use a trade query or trade snapshots, and a few other features, as documented in the comments.
package com.findur.oc.eod;
import java.util.ArrayList;
import java.util.List;
import com.findur.oc.util.*;
import com.olf.embedded.*;
import com.olf.openjvs.*;
import com.olf.openrisk.*;
@ScriptCategory({EnumScriptCategory.Generic})
public class CreateSimTasks extends AbstractGenericScript {
/**
* The sim task snapshot type to use. This must match the configuration of the sim task definition
* <p>E.g. "EOD" or "EOM"
*/
private static final String SNAPSHOT_TYPE = "EOD";
/**
* The sim task snapshot date to use. For the sim task to complete successfully, there must be a saved snapshot with this date.
* <p>It is typical to use symbolic dates such as "0d" (for today)
*/
private static final String SNAPSHOT_DATE = "0d";
/**
* Determines whether to create the Sim Task using a transaction source of TradeSnapshot (true) or Query (false)
*/
private static final boolean USE_TRADE_SNAPSHOT = true;
/**
* The prefix used for the query and sim task names
* E.g. "EOD " will lead to a query called "EOD MyPortfolioName"
*/
private static final String EOD_QUERY_PREFIX = "EOD ";
/**
* The prefix used to name the sim task
*/
private static String simTaskNamePrefix = EOD_QUERY_PREFIX;
/**
* The market data set to use for the sim task
* E.g. "Closing", "Universal"
*/
private static final String MARKET_DATA_CONFIG = "Closing";
/**
* The name of the saved simulation definition (selection of scenarios, results, etc) for the sim task
* E.g. "EOD", "Risk"
*/
private static String simDefinitionName = "EOD";
/**
* The reval type to use for the sim task
* E.g. "EOD", "Accounting", "Risk"
*/
private String revalType = "EOD";
/**
* The prior day reval type to use for the sim task
* E.g. "EOD", "Accounting", "Risk"
*/
private final String priorRevalType = revalType;
private Session session;
private List<String> exceptions = new ArrayList<String>();
private MessageLog messageLog;
private String errorLogFile = this.getClass().getSimpleName() + ".log";
public CreateSimTasks() {
}
public CreateSimTasks(String errorLogFile) {
this.errorLogFile = errorLogFile;
}
public Table execute(final Session session, final ConstTable table) {
this.session = session;
messageLog = new MessageLog(session, errorLogFile);
messageLog.info("START [" + errorLogFile + "] ");
try {
int[] portfolioIds = getPortfoliosId();
messageLog.info("Checking [" + portfolioIds.length + "] portfolios");
for (int portfolioId: portfolioIds) {
Portfolio portfolio = null;
try {
portfolio = session.getStaticDataFactory().getReferenceObject(Portfolio.class, portfolioId);
if (portfolioHasTrades(portfolio)) {
//Create the EOD Sim Task for the portfolio
if (!eodSimTaskExists(portfolio)) {
createEODSimTask(portfolio);
}
}
} finally {
PluginHelper.dispose(portfolio);
}
}
} catch (Exception e) {
messageLog.logException(e);
throw new OpenRiskException(e);
} finally {
messageLog.info("END [" + errorLogFile + "] ");
emailResults();
if (exceptions.size() > 0) {
throw new OpenRiskException(this.getClass().getSimpleName() + " reported [" + exceptions.size() + "] error" + MessageLog.pluralize(exceptions.size()));
}
}
return null;
}
private int[] getPortfoliosId() {
String sql = "SELECT DISTINCT p.id_number portfolio_id"
+ "\nFROM portfolio p"
//Exclude portfolios that already have a sim task
+ "\nWHERE p.id_number NOT IN ("
+ "\n SELECT q.ival"
+ "\n FROM sim_task st"
+ "\n JOIN trade_snapshot_def tsd ON tsd.snapshot_name = st.trade_snapshot_name"
+ "\n JOIN query_rec q ON q.id = tsd.query_id"
+ "\n AND q.table_name = 'ab_tran'"
+ "\n AND q.field_name = 'internal_portfolio'"
+ "\n JOIN portfolio p ON p.id_number = q.ival"
+ "\n)"
//Exclude portfolios that have no validated trades
+ "\nAND p.id_number IN ("
+ "\n SELECT DISTINCT internal_portfolio"
+ "\n FROM ab_tran"
+ "\n WHERE tran_status = " + EnumTranStatus.Validated.getValue()
+ "\n)";
Table table = null;
try {
table = session.getIOFactory().runSQL(sql);
if(table != null && table.getRowCount() > 0) {
return table.getColumnValuesAsInt("portfolio_id");
}
return new int[0];
} finally {
PluginHelper.dispose(table);
}
}
/**
* Identifies whether the portfolio has trades for today.
* <p>When creating a sim task, if the portfolio has no trades then the Trade Snapshot Type field cannot be set.
* <p>We will not create the sim task definition until there are trades in the portfolio
*/
private boolean portfolioHasTrades(Portfolio portfolio) {
String queryName = EOD_QUERY_PREFIX + portfolio.getName();
try {
int numTrades = QueryHelper.getObjectIds(session, queryName).length;
if (numTrades > 0) {
return true;
}
} catch (Exception e) {
//If there is an exception then assume no trades. The query is probably missing so the portfolio is incompletely setup.
messageLog.error(e.getMessage());
exceptions.add(e.getMessage());
}
return false;
}
/**
* Checks if the Sim Task already exists
*/
private boolean eodSimTaskExists(Portfolio portfolio) {
boolean retVal = false;
String simTaskName = simTaskNamePrefix + portfolio.getName();
SimTask simTask = null;
try {
simTask = SimTask.getByName(simTaskName);
if (simTask == null) {
messageLog.info( "EOD Sim Task does not exist for portfolio: " + portfolio.getName());
retVal = false;
} else {
retVal = true;
}
} catch (OException e) {
messageLog.info( "EOD Sim Task does not exist for portfolio: " + portfolio.getName());
messageLog.info( e.getMessage());
retVal = false;
}
if (simTask != null) {
simTask.dispose();
}
return retVal;
}
/**
* Creates a new EOD Sim Task called eodQueryPrefix + portfolio.getName();
*/
private void createEODSimTask(Portfolio portfolio) {
int retVal = 0;
String newTaskName = simTaskNamePrefix + portfolio.getName();
String newQueryName = EOD_QUERY_PREFIX + portfolio.getName();
SimTask simTask = null;
try {
simTask = SimTask.createNew();
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_NAME, newTaskName);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_REVAL_TYPE, revalType);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_DEF, simDefinitionName);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_MARKET_DATA_CFG, MARKET_DATA_CONFIG);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_PRIOR_REVAL_TYPE, priorRevalType);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SAVE_AFTER_EXE, "Yes");
if (USE_TRADE_SNAPSHOT) {
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_QUERY, newQueryName);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_TRADE_SOURCE, "Sim Trade Snapshot");
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_TRADE_SNAPSHOT_REF_NAME, newQueryName);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_TRADE_SNAPSHOT_TYPE, SNAPSHOT_TYPE);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_TRADE_SNAPSHOT_DATE, SNAPSHOT_DATE);
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_QUERY, newQueryName);
} else {
setSimTaskField(simTask, SIMTASKF_FIELD.SIMTASKF_SIM_QUERY, newQueryName);
}
} catch (Exception e) {
messageLog.error(e.getMessage());
exceptions.add("Portfolio [" + portfolio.getName() + "], SimTask [" + newTaskName + "], " + e.getMessage());
//Don't throw exception. The errors collection will be examined later for exceptions.
return;
}
try {
retVal = simTask.save();
if (retVal < 1) {
exceptions.add("Failed to save Sim Task for Portfolio [" + portfolio.getName() + "], " + e.getMessage());
}
messageLog.info( "Created sim task for portfolio [" + portfolio.getName() + "]");
} catch (Exception e) {
messageLog.error(e.getMessage());
exceptions.add("Portfolio [" + portfolio.getName() + "], SimTask [" + newTaskName + "] save operation failed. " + e.getMessage());
}
simTask = null;
}
/**
* Helper methed to set the sim task field and throw exceptions
*/
private void setSimTaskField(SimTask simTask, SIMTASKF_FIELD simtaskfName, String string) throws OException {
if (simTask.setField(simtaskfName.toInt(), string, null) != 1) {
throw new OpenRiskException("Failed to set sim task field [" + simtaskfName.toString() + "] to value [" + string + "]");
}
}
/**
* Overrides the sim definition name
*/
public void setSimDefinitionName(String newSimDefinitionName) {
simDefinitionName = newSimDefinitionName;
}
/**
* Overrides the reval type
*/
public void setRevalType(String newRevalType) {
revalType = newRevalType;
}
/**
* Overrides the prefix of the sim task name
*/
public void setSimTaskNamePrefix(String newTaskNamePrefix) {
simTaskNamePrefix = newTaskNamePrefix;
}
}