Stream a ServiceM8 account

A ServiceM8 stream lands a ServiceM8 account's jobs and quotes, the lines and payments behind each invoice, the schedule, clients, staff time, forms and assets in your warehouse, as often as every 15 minutes. After the first run, each run reads only what changed in ServiceM8 since the last.

ServiceM8 is a published source, like Postgres and Bite: sanda wrote the code that reads it and runs it as its own, so there is no connector for sam to write or for anyone to approve. You type an API key the account's Business Owner makes in ServiceM8, choose what to read, and set a schedule.

Before you start

• The Business Owner makes the key. In ServiceM8, only the Business Owner can make an API key. It is under Settings · API Keys (the button may read API Access). Make an API key there, not a Plugin API key (one that starts smk-): those are ServiceM8's older kind, and its API doesn't accept them. sanda only reads, so Read Only access is enough. Give the key a name such as sanda, so you can tell it apart from other keys later.
• One account at a time. Each key reads one ServiceM8 account. A business with two accounts connects each one and makes a stream for each.
• Only owners and admins connect a key and make a stream on it in sanda, because sanda reads the account on the whole workspace's behalf.

Connect an account

sanda keeps the key encrypted, only ever sends it to ServiceM8, and never shows it again, in the console or to sam. Type it again replaces the key (after you make a new one in ServiceM8, say), and every stream on the account carries on. A key for a different ServiceM8 account is refused there, because a stream keeps the account it reads. Disconnect forgets the key in sanda. To stop it working in ServiceM8 as well, revoke it under Settings · API Keys.

Make a stream

What it reads

Jobs, job contacts, clients, client contacts and staff hold people's names, phone numbers, email addresses or home addresses, and the editor says so under each. Untick them if your warehouse shouldn't hold them.

What changed since the last run

The first run reads each list whole. After that, a run asks ServiceM8 for every record changed after a date: the date of the newest change the last run saw, less one day. So each run reads a day or two of changes again. A record read twice is still one row, and one that hasn't changed costs nothing in your warehouse. Reading a little again means a change is never missed, whatever ServiceM8's clock did, including the hour that repeats when daylight saving ends.

Jobs, quotes and invoices

A quote is a job whose status is Quote, and a work order one whose status is Work Order: ServiceM8 keeps them in one list, so they land in one table. ServiceM8 has no separate invoice record either: a job's invoice is its invoice_date, invoice_sent and total_invoice_amount, with its lines (labour included) in job materials and the money received in job payments. Each line and payment names its job in job_uuid.

The schedule and staff time

Job activities are both the bookings on the schedule and the time staff recorded on a job, told apart by activity_was_scheduled and activity_was_recorded. Job allocations are jobs given to someone without a booked time. Staff clock events are each clock on, clock off and lunch break.

Job admin time

ServiceM8 records no time of change for office time logged against a job, so a run can't ask for what changed. Instead, the first run reads all of it, and after that, once a day, the stream reads the last 60 days again. An entry corrected inside those days lands corrected, and one removed is marked _deleted_at. An entry older than that keeps what it last said.

Tax rates and the account

Both are a handful of rows, read whole every run. The account is the ServiceM8 account itself: its timezone_name is the time zone ServiceM8's times are in (with one exception: a form response's timestamp is UTC), and its currency the currency of every amount.

Left out

ServiceM8 also offers these, which a ServiceM8 stream doesn't read yet: job notes, tasks, checklists and job templates, attachments (such as the quote and invoice PDFs), suppliers, locations, allocation windows and security roles.

A member of staff's last known position (lat, lng and geo_timestamp) and the job they are travelling to (navigating_to_job_uuid and its times) never land.

What lands in your warehouse

Every field becomes a column, and the whole record is kept in _record, as for every pull. A few things are worth knowing when you model them:

• Times are the account's own time, with no time zone. ServiceM8 writes its times as the wall-clock time where the account is, so these columns are timestamps without a zone. The account table's timezone_name says which zone: in a view, edit_date at time zone 'Australia/Sydney' gives the instant. A time ServiceM8 never set, which it sends as 0000-00-00 00:00:00, is empty in its column and kept in _record.
• A form response's timestamp is UTC. When a form was completed is the one time ServiceM8 writes in UTC, so that column is a timestamp with a time zone: it already holds the instant. In a view, "timestamp" at time zone 'Australia/Sydney' gives the time on the account's clock, to set beside its other times. A form response's edit_date is the account's own time, like every other table's.
• Amounts are numbers. ServiceM8 sends money and quantities as text. They land as numbers in total_invoice_amount on jobs, in quantity, price, cost, displayed_amount and displayed_cost on job materials, in price and cost on materials, in amount on job payments and in amount (a percentage) on tax rates. A job material's price and cost are without tax, and its displayed_amount_is_tax_inclusive says whether displayed_amount includes it.
• Deleted records stay, with active 0. Deleting something in ServiceM8 marks it inactive rather than removing it, and ServiceM8 still lists it, so its row keeps coming in with active 0. Leave those out in a view where you count jobs or revenue.
• Ids are ServiceM8's uuids. Every table is keyed by uuid, and a column ending in _uuid names a row of another table: company_uuid a client, staff_uuid a member of staff, category_uuid a job category.
• Badges are uuids. A job's or client's badges holds the uuids of its badges, each a row of badges.

Change a stream

Edit, on the stream's page, changes everything but the account and the warehouse. Something you take off keeps its table, with what it landed. Load everything again on the stream's page reads every list whole on the next run. A large account's first read, or a load from the start, can take several runs, each carrying on from the last.

Remove ServiceM8 data

ServiceM8's terms ask that its data be deleted when you stop using it, or when someone asks. To remove an account's data from sanda:

When something goes wrong

A failed run counts toward the stream pausing itself, as any stream's does. A run that stopped for the daily allowance, or was skipped while it rests, doesn't. See runs.

Limits

• ServiceM8 allows each account 180 requests a minute and 20,000 a day. A run makes at most 2 requests a second, which keeps it inside the minute's allowance, and a stream runs at most every 15 minutes, which keeps its everyday runs well inside the day's. A large account's first read, or Load everything again, can spend more of the day.
• Each request returns up to 1,000 records: ServiceM8 sets the page size.
• One run lands at most 20,000,000 records, so a large account's first read is several runs.

A ServiceM8 stream is billed like any pull: a charge for each run that does its work, and the rows it lands past what your edition includes. See what streams cost.