# Stream a Simpro build

> Land a Simpro build's jobs, quotes, leads, invoices, schedules and job costs in your warehouse every few minutes, connected through Simpro's own sign-in.

A Simpro stream lands a Simpro build's jobs, quotes, leads, invoices, schedules and job costs in your warehouse, as often as every few minutes, with the customers, sites, staff and lists they refer to. Each run reads only what changed since the last run, and Simpro's own work in progress report, kept day by day.

Simpro is a **published source**, like [Bite](https://docs.sanda-os.com.au/streams/bite): sanda wrote the code that reads it and runs it as its own, so there is no connector for sam to write or for anyone to approve. You connect your build through Simpro's own sign-in, choose the company and what to read, and set a schedule.

:::note
Simpro is in the catalogue once connecting Simpro is switched on for your console. If **Choose a source** doesn't list it, ask sanda support.
:::

## Before you start

- **Know your Simpro address.** It is the address you sign in to Simpro at, such as `yourcompany.simprosuite.com` or `yourcompany.simprocloud.com`. The first part alone will do for a `simprosuite.com` build.
- **Sign in as someone who can see the data.** Simpro has no separate permissions for sanda: a stream sees what the Simpro user who connected the build sees, through their security group. Connect as someone whose security group can see jobs, quotes, invoices, schedules and the rest of what you want to read. Something their security group hides is skipped, and the run says so.
- **One company at a time.** A Multi-company build holds several companies, and each stream reads one of them. Make a stream for each company you want. A build with one company has just the one, numbered 0.
- **In sanda, only owners and admins connect a build** and make a stream on it, because sanda reads the build on the whole workspace's behalf.

## Connect a build

:::steps
1. **Open Pull.** Go to **Data · Streams**, open the **Pull** tab and press the **Simpro** card under **Choose a source**. The new pull's editor opens on **Connect a Simpro build** when none is connected yet. Once a build is connected, **Connect a build** beside Simpro under **Connected** adds another.
2. **Type your Simpro address** under **Simpro address**, then press **Continue to Simpro**. sanda checks the address first: it must end in `simprosuite.com` or `simprocloud.com`.
3. **Sign in to Simpro**, on your own build's sign-in page, and allow sanda when Simpro asks. If you don't, nothing is connected.
4. **Back in sanda**, a line at the top of the page says the build is connected, and it is listed under Simpro in **Connected** by its address. If you started from a new stream, you are back in its editor with the build chosen.
:::

sanda keeps the connection encrypted, and it is never shown, in the console or to sam. **Connect again** connects the build again: sign in to the same build, and every stream on it carries on. A connection nobody uses stops working after 14 days, so sanda renews each connected build's connection before then, whether or not a stream ran.

**Disconnect** forgets the connection in sanda. Simpro offers no way for sanda to end it from here, so to remove sanda's access in Simpro too, remove it there. The build's streams then wait until it is connected again.

## Make a stream

:::steps
1. **Start it.** Press the **Simpro** card under **Choose a source**, or **New pull** and then **Simpro**.
2. **Choose when it runs.** **When it runs** comes first: how often, within which hours, on which days, with every run drawn out and the cost a month beneath. See [when a stream runs](https://docs.sanda-os.com.au/streams/schedules).
3. **Name it.** Its tables are named after it: a stream called `simpro` lands jobs in `raw.simpro__jobs`.
4. **Choose the build**, and, if your workspace has more than one warehouse, the warehouse it lands in. Both are chosen once.
5. **Choose what it reads.** Everything but **Job cost centre totals** is ticked. Untick what you don't need.
6. **Choose the company.** **Company** lists the companies the user who connected the build can reach. Most builds have one. If the list doesn't load, **Type it instead** takes the company's number. A stream keeps its company.
7. **Read history from** is optional: the date the first run starts from for jobs, quotes, leads, invoices, schedules, job costs, purchase orders and contractor invoices. Leave it empty to read the last 730 days.
8. **Press Save pull.** Saving reads nothing yet: press **Run now**, or wait for its schedule.
:::

## What it reads

| Tick | Lands in | Each run reads |
|---|---|---|
| **Jobs** | `raw.<stream>__jobs` | The jobs that changed since the last run |
| **Quotes** | `raw.<stream>__quotes` | The quotes that changed since the last run |
| **Leads** | `raw.<stream>__leads` | The leads that changed since the last run |
| **Invoices** | `raw.<stream>__invoices` | The invoices that changed since the last run |
| **Schedules** | `raw.<stream>__schedules` | The schedules that changed since the last run |
| **Job cost centres** | `raw.<stream>__job_cost_centers` | The job cost centres that changed since the last run |
| **Job cost centre totals** | `raw.<stream>__job_cost_center_totals` | The job cost centre totals that changed since the last run |
| **Work in progress** | `raw.<stream>__wip_cost_to_complete` | Today's work in progress, at most once every 60 minutes |
| **Purchase orders** | `raw.<stream>__vendor_orders` | The purchase orders that changed since the last run |
| **Contractor invoices** | `raw.<stream>__contractor_invoices` | The contractor invoices that changed since the last run |
| **Company customers** | `raw.<stream>__customer_companies` | The company customers that changed since the last run |
| **Individual customers** | `raw.<stream>__customer_people` | The individual customers that changed since the last run |
| **Sites** | `raw.<stream>__sites` | The sites that changed since the last run |
| **Employees** | `raw.<stream>__employees` | The employees that changed since the last run |
| **Contractors** | `raw.<stream>__contractors` | The contractors that changed since the last run |
| **Staff** | `raw.<stream>__staff` | The whole list |
| **Catalogue items** | `raw.<stream>__catalog_items` | The catalogue items that changed since the last run |
| **Customer assets** | `raw.<stream>__customer_assets` | The customer assets that changed since the last run |
| **Cost centres** | `raw.<stream>__cost_centers` | The whole list |
| **Project tags** | `raw.<stream>__project_tags` | The whole list |
| **Project custom fields** | `raw.<stream>__project_custom_fields` | The whole list |

Each table is keyed by Simpro's own id. The editor marks the ones that land people's contact details.

### Jobs, quotes, leads and the rest that change

Each run asks Simpro for what changed since the newest change the last run saw, less 10 minutes, so a change made while the last run was reading isn't missed. A record read twice is still one row, and one that hasn't changed costs nothing. Simpro lists them in order of their id, and each request asks for up to 250 records after the last one the request before saw, so a record that changes while a run reads can't push another out of the way.

A first run reads jobs, quotes, leads, invoices, schedules, job cost centres, purchase orders and contractor invoices changed since the history date. It reads all of the customers, sites, employees, contractors, catalogue items and customer assets, whatever their age, so every job's customer and site has its row.

### Job cost centre totals

Simpro's list of job cost centres holds their names and ids but none of their figures. **Job cost centre totals** asks Simpro for each changed cost centre on its own: its stage, start and end dates, percent complete, what has been claimed, and its cost and margin totals. That is one request per cost centre, so a first run on a large build takes a while, which is why it is unticked to begin with. Each row also carries the `jobid` and `sectionid` of the job and section it sits under.

### Work in progress

**Work in progress** is Simpro's cost to complete report: for each job, its contract total, what has been claimed to date, cost to date, cost to complete, percent complete and margins. sanda asks for it at most once every 60 minutes, as at that day's date in the company's own time zone, which it asks Simpro for first. If Simpro doesn't name a zone, the day is Sydney's. sanda keeps each day's report: a job's row is keyed by its job id and `as_at`, the day it was taken, and a later read on the same day updates that day's rows, so each day keeps its last read. Over time the table is a daily history of each job's work in progress. A job that leaves the report during a day keeps the row an earlier read gave it that day. sanda asks for it with Simpro's own defaults for variations, committed costs and overheads.

### Staff and the lists

Cost centres, project tags and project custom fields carry no date of their own, so each run reads them whole. Staff do carry one, in `datemodified`, but the list is short, so each run reads it whole too and notices someone taken off it. A row the list no longer has is marked `_deleted_at` and kept, and reports and answers stop counting it.

### Left out

- **Pay and personal details.** Employees' pay rates, bank details, tax numbers, birth dates, home addresses and emergency contacts, contractors' rates, bank details, tax numbers and addresses, and customers' bank details are never kept. Simpro sends some of them with a record sanda does read: a contractor invoice comes with its contractor's address, contact details and bank account, so sanda keeps only the contractor's id and name from it.
- **Descriptions and notes.** The free text on jobs, quotes, leads, sites and invoices stays in Simpro. So do the descriptions that come with an invoice's jobs and its recurring invoice and with a contractor invoice's lines, and the notes on a customer's profile.
- **Timesheets.** Simpro lists them one employee at a time, so reading them means a request for every employee on every run. **Schedules** holds the scheduled time instead.
- **Job logs and stock on hand** are not read. Stock is listed one storage location at a time, and sanda can't confirm the job log can be asked for only what changed.

## What lands in your warehouse

Every field becomes a column, and the whole record is kept in `_record`, as for every [pull](https://docs.sanda-os.com.au/streams/in#where-the-records-land). A few things are worth knowing when you model them:

- **Every table has `company_id`**: the company in the build its records came from, so two companies' streams can be joined without their ids colliding.
- **Money is a JSON object** where Simpro sends one: a job's `total` holds `ExTax`, `Tax` and `IncTax`, and its `totals` the estimated, revised, actual and committed costs, margins and invoiced value. A catalogue item's prices, a customer's amount owing and the work in progress figures are plain numbers.
- **Dates are your build's own calendar days**, such as a job's `dateissued`. `datemodified` is a time with your build's own offset, so it compares correctly whatever the warehouse's time zone.
- **Ids are numbers**, and the customer, site, salesperson and technicians on a job, quote or invoice are JSON objects holding each one's `ID`, which names a row in that list's table. A job's `tags` hold its project tags' ids and names.
- **Deleted records.** Simpro lists what changed, not what was deleted. A first run, and every **Load everything again**, reads each list in full (from the history date, for the lists that have one) and marks a row Simpro no longer has with `_deleted_at`. Between those, a record deleted in Simpro keeps its row.

## Change a stream

**Edit**, on the stream's page, changes everything but the build, the company and the warehouse. Something you take off keeps its table, with what it landed. A new **Read history from** date takes effect the next time everything is read from the start: press **Load everything again** on the stream's page.

## When something goes wrong

| What you see | What to do |
|---|---|
| Simpro didn't let the user who connected this build read something: their security group doesn't allow it | The run skipped that one and read the rest. Take it off the stream with **Edit**, or ask a Simpro admin to let that user's security group see it, or connect the build again as a user who can |
| Simpro didn't let the build sanda reads with see something, though it still signs in | As above: the user who connected the build can sign in but can't see that part of Simpro |
| Simpro no longer accepts sanda's connection to the build | The user who connected it may have left, or lost access. The stream waits, rather than failing again and again. Press **Connect it again** on the stream's page, or **Connect again** beside the build under **Connected**, and sign in to the same build |
| That isn't a Simpro address sanda can connect to | Type the address you sign in to Simpro at. It ends in `simprosuite.com` or `simprocloud.com` |
| Simpro asked whether to allow sanda and the answer was no | Connect again, and allow sanda when Simpro asks |
| Simpro refused sanda's request for something, in Simpro's own words | Something in the build stops Simpro answering that list. The words after it say what. If they don't help, sanda support can see why |
| Simpro asked sanda to slow down | Nothing to do: the run pauses for a second, as often as Simpro asks, then carries on |
| Simpro didn't answer, or answered with an error of its own | Nothing to do: the run tries again, and the next run carries on from the last page that landed |

A failed run counts toward the stream pausing itself, as any stream's does. See [runs](https://docs.sanda-os.com.au/streams/runs).

## Limits

- Simpro allows each integration 10 requests a second on one build, shared by everything sanda does with that build. A run makes at most 4 requests a second, and each request asks for up to 250 records.
- A connection nobody uses for 14 days stops working. sanda renews it before then.
- One run lands at most 20,000,000 records, so a long history is several runs.

A Simpro stream is billed like any pull: a charge for each run that does its work, and the rows it lands past what your edition includes. See [what streams cost](https://docs.sanda-os.com.au/streams#what-streams-cost).

:::links
- [Pull](https://docs.sanda-os.com.au/streams/in): Every way records come into your warehouse on a stream.
- [Stream graphs](https://docs.sanda-os.com.au/streams/graphs): Model what a stream lands, then send the result out, every time it lands.
- [Runs](https://docs.sanda-os.com.au/streams/runs): What each run did, and what to do when one fails.
:::
