Object Reference

This document contains a description of the objects available via the LeadConduit Batch API.

Batch

A Batch is the primary datapoint in LeadConduit Batch. When a lead or record is submitted to LeadConduit Batch, the batch ID is specified and the lead is processed according to the instructions in that batch.

Property Description Type
id The ID of the batch which uniquely identifies it BSON ID
name The name of this batch that is displayed throughout LeadConduit's UI String
file_name_template The template by which each batch file will be named Template
file_name_timezone The template's time variables will be based on this timezone, or UTC if empty String
columns The fields that will be collected in each batch file Array of Strings
schedule A cron expression or quantity of time by which the batch file will be created Schedule Expression
outbound Turns sending batch files on and off. Boolean
inbound Turns recording submitted data on and off. Boolean
format Specific formatting of the batch file Format
delivery Method of delivering batch files and its configuration Delivery
success_notification_emails Email addresses to receive notification on successful batch file delivery Array of Strings
error_notification_emails Email addresses to receive notification on failed batch file delivery Array of Strings
send_notification_on_empty Whether or not to send a success notification if the scheduled batch file is empty Boolean

^ back to top

File Name Template

When a batch file is created, the file name template is used to create a unique filename for the collection of records. A selection of variables are available to be used inside double brackets ({{}}, i.e. {{YYYY}}). These variables are mostly references to the particular time the file was generated, along with the availability of the batch ID as a variable. For example, the file name template {{YYYY}}-{{MM}}-{{DD}}.csv could yield 2016-12-25.csv.

Variable Description
YYYY Year, 4-digit representation
YY Year, 2-digit representation
M Month [1-12]
MM Month, zero-padded 2-digit representation
MMM Month Name, shortened
MMMM Month Name, full name
Q Quarter, number only
Qo Quarter with ordinal indicator
D Day [1-31]
DD Day, zero-padded 2-digit representation
dd Day of Week, 1-2 characters
ddd Day of Week, shortened name
dddd Day of Week, full name
W Week of the year
WW Week, zero-padded 2-digit representation
A Period of AM/PM in uppercase
a Period of am/pm in lowercase
H Hour, 24-hour representation [0-23]
HH Hour, 24-hour representation, zero-padded
h Hour, 12-hour representation [1-12]
hh Hour, 12-hour representation, zero-padded
m Minute [0-59]
mm Minute, zero-padded
s Second [0-59]
ss Second, zero-padded
z Time Zone by abbreviation
Z Time Zone by +/- GMT
ZZ Time Zone by +/- GMT, zero-padded
X Unix Timestamp in Seconds
x Unix Timestamp in Milliseconds

^ back to top

Schedule Expression

Leads are collected and sent in batch files determined by the batch schedule. This schedule can take two forms: a cron expression, or an amount of time followed by a unit of time.

Cron expressions are expressed as follows:

 ┌───────────── min (0 - 59)
 │ ┌────────────── hour (0 - 23)
 │ │ ┌─────────────── day of month (1 - 31)
 │ │ │ ┌──────────────── month (1 - 12)
 │ │ │ │ ┌───────────────── day of week (0 - 6) (Sunday to Saturday;
 │ │ │ │ │                                         7 is also Sunday)
 │ │ │ │ │
 │ │ │ │ │
 * * * * *

Multiple times per interval are allowed using the */x syntax or x,y,z syntax.

Schedules based on intervals can be expressed in the following way:

(amount of time) (unit of time)

Valid units of time are: seconds, minutes, hours, days, weeks, and years. For example, 15 minutes or 1.5 hours would be acceptable intervals.

^ back to top

Format

A batch file is a file of tabular data stored in plain text. The most common form is as a CSV (comma-separated values) file. To allow for some variance in the platform consuming the data, there are several settings that modify the file returned by the batch configuration.

Property Description Type
integration Details about the specific integration that is being used for formatting Integration
use_compression Whether to compress the CSV file into a ZIP file Boolean
pgp Object that provides details about using PGP encryption PGP

^ back to top

Format Integration

The format integration is a particular integration designed specifically for altering the format of the data stored in the database. By default, the batch-csv.format integration is used, but each integration has its own unique set of config properties.

Property Description Type
module_id The ID of the integration being used, defaults to batch-csv.format String
version Which version of the integration to use. This is a semantic version number, where * (default) means "latest" String or Number
config An object with properties that provide configuration settings for the integration Object

When using the default integration, the following properties are used on the config object:

Property Description Type
row_separator This determines whether rows are separated by UNIX (\n) or Windows (\r\n) standards String ('unix' or 'windows')
use_header Whether to include a header row with field names Boolean
delimiter Character used to separate individual fields of data String (',', '|', or '\t')

^ back to top

PGP

PGP encryption is supported when formatting batch files. This allows for securing data in your files using a key. This operation occurs after an integration converts the data into a file and before standard ZIP compression.

The PGP configuration object supports the following properties.

Property Description Type
enabled Whether PGP encryption is enabled or not Boolean
armor Whether the file will be converted to base64 or left as binary, defaults to true (base64) Boolean
compression The algorithm that should be used for compression, either 'uncompressed', 'zip', or 'zlib', defaults to 'uncompressed' String
public_key The public key that will be used for encryption. This must currently be an RSA key. If a key is not provided, a key pair will be generated and the private key will be returned in the creation of the key. String
signing_private_key The private key that will be used for signing. This is not required, but helps to ensure that the encrypted file was not created by another party. If no key is provided, ActiveProspect's default signing key will be used String
signing_password The password used to decrypt the private signing key, if needed. String

^ back to top

Delivery

The batch files are accessible to download via the LeadConduit user interface, but the delivery method determines how the files will be sent at the scheduled time. Regardless of whether or not there have been any leads collected in the time since the last delivery, a file will be sent. If no leads were collected, an empty file will be sent.

The currently supported methods include: email, FTP, SFTP, FTPS, and Google Cloud Storage. Below is a list of all supported properties, although each method only requires and/or uses a limited set of those properties. More details on the properties can be found in the subsections.

Property Description Type
method The method of delivery, regardless of other defined properties String
subject An email subject String
to Email addresses to receive delivery, separated by comma Array of Strings
hostname The server's IP or domain name on which to store the leads (all methods except email and Google Cloud) String
port The port on which to connect to the server, defaults vary based on method Number
remote_path The folder on the server in which to store the batch file String
username A username if required for authentication String
password A password if required for authentication String or Boolean
allow_self_signed Whether or not to allow the batch file to be uploaded to a server with a self-signed SSL certificate Boolean
project_id A project identifier String
bucket A bucket identifier String
client_email The client email to be used as credentials String
client_private_key The private key to be used as credentials String

Although the password field must be stored in the API in a manner that allows it to be used for authentication to another server, the field itself cannot be retrieved from the API. When the password is set, the API will return the boolean value of true to note that is has been set. If a value other than true is submitted when updating a batch, that value will be used to set the new value of the password field. If true is submitted, the password will remain unchanged.

^ back to top

Email Delivery

The most basic means to receive batch files is by email. The leads will be delivered via the email address delivery@yourlead.info. Please be sure to whitelist this email address to ensure proper delivery. The following parameters are supported for this delivery method:

Property Description Type
subject An email subject, with a default referring to the batch and number of leads String
to Email addresses to receive delivery, separated by comma Array of Strings

^ back to top

FTP Delivery

This method is supported for older servers and configurations that don't allow SFTP or FTPS. If there is an option to use one of the other two methods, it is advised to do so.

Property Description Type
hostname The server's IP or domain name on which to store the leads String
port The port on which to connect to the server, defaults to 21 Number
remote_path The folder on the server in which to store the batch file String
username A username if required for authentication String
password A password if required for authentication String

^ back to top

FTPS Delivery

This method is similar to FTP, with the added support of TLS and SSL protocols.

Property Description Type
hostname The server's IP or domain name on which to store the leads String
port The port on which to connect to the server, defaults to 21. Another common for for FTPS is 990 Number
remote_path The folder on the server in which to store the batch file String
username A username if required for authentication String
password A password if required for authentication String
allow_self_signed Whether or not to allow the batch file to be uploaded to a server with a self-signed SSL certificate Boolean

^ back to top

SFTP Delivery

SSH File Transfer Protocol is similar to SCP with the added support for a wider range of file operations. This is not FTP over SSH. The Batch API currently supports authentication via password and keyboard-interactive.

Property Description Type
hostname The server's IP or domain name on which to store the leads String
port The port on which to connect to the server, defaults to 22 Number
remote_path The folder on the server in which to store the batch file String
username A username if required for authentication String
password A password if required for authentication String

^ back to top

Google Cloud Storage Delivery

Deliver a batch file to a Google Cloud Storage bucket. It uses client_email and private_key from the GCS Service Account Key to authenticate. The method of delivery should be defined as google-cloud-storage.

Property Description Type
remote_path The folder on the server in which to store the batch file String
project_id A project identifier String
bucket A bucket identifier String
client_email The client email to be used as credentials String
client_private_key The private key to be used as credentials String

^ back to top

Batch File

A Batch File is the document in LeadConduit Batch that represents a particular set of leads. When a batch is scheduled for delivery, a batch file is generated automatically that is similar to a recipe for generating an actual file. Batch Files can also be created via the LeadConduit Batch API.

Property Description Type
id The ID of the batch file which uniquely identifies it BSON ID
batch The full batch configuration that was used to generate this file Batch
name The name of this batch file (used for filename) String
minId The chronological starting point of leads to collect for this batch file BSON ID
maxId The chronological ending point of leads to collect for this batch file BSON ID
count The number of leads collected in this file Number
error Any error that has been recorded when attempting to deliver this file String
created_at The creation time of the original batch file Timestamp
updated_at When this batch file was updated if it originally erred Timestamp
started_at When this batch file was initially attempted to be delivered Timestamp
finished_at When this batch file was finished with its delivery, erred or not Timestamp
expires_at When this batch file will be deleted, based on account data retention settings Timestamp

^ back to top

Batch Record

A batch record is a basic JSON document of data that acts as an individual entry in a batch file. The key-value pairs provided in the JSON document should correspond to the columns specified in the batch configuration.

^ back to top

BSON ID

A 24 character hexidecimal string identifier. An identifier is guaranteed to be unique across all objects of the same type.

^ back to top

Timestamp

In LeadConduit, timestamps on objects are always provided in ISO8601 format in the UTC timezone. The format is YYYY-MM-DDTHH:MM:ss.SSSZ. For example 2015-07-28T19:14:44.033Z.

^ back to top