Apify — dataset/getItems

When to Use

Post-async-run data retrieval: After an actor/run (with waitForFinish: false) completes and the defaultDatasetId is known, use this node to fetch the scraped items — decoupling the run trigger from the data consumption.
Paginated large dataset processing: An actor produces 10,000+ items. Use Offset and Limit inside a Loop to process items in batches of 100–500, feeding each batch into downstream database upsert or transformation nodes.
Field-projected data loading: An actor returns wide records with 30+ fields; the downstream database schema requires only 5. Use Fields to project only the needed columns, reducing payload size and mapping complexity.
Differential data processing: Re-read an existing dataset from a past actor run (dataset ID stored in a database) to compare against current data — useful for change detection, delta loads, and incremental processing without re-running the actor.
Multi-dataset aggregation: In a Loop node over an array of dataset IDs (e.g. from multiple actor runs), fetch and merge items from each dataset to build a consolidated output for reporting or analysis.

Dataset ID sources: Obtain the DatasetId from the defaultDatasetId field returned by any run operation (actor/run, actor-task/run, actor-run/get, actor/getLastRun). Alternatively, find dataset IDs in the Apify Console under Storage → Datasets. Named datasets can be referenced by their ID — dataset names are not supported directly.

Configuration

Connection

Field	Required	Description
`credentialId`	Required	BizFirstAI credential ID holding the Apify API token.
`baseUrl`	Optional	Apify API base URL. Defaults to `https://api.apify.com`.

Operation

Field (config key)	Required	Description
`resource`	Required	Must be `"dataset"`.
`operation`	Required	Must be `"get-items"`.
`datasetId`	Required	The Apify dataset to read. Obtain from `defaultDatasetId` in any run operation output. Supports BizFirst expressions.
`offset`	Optional	Number of items to skip. Default `0`. Must be non-negative. Use for pagination: page 2 with limit 100 uses `offset: 100`.
`limit`	Optional	Maximum items to return. Default `50`. Valid range: 1–10000.
`fields`	Optional	Comma-separated list of field names to include in each returned item. Omit to return all fields. Example: `"title,url,price,inStock"`.
`omit`	Optional	Comma-separated list of field names to exclude from each returned item. Applied after `fields` projection. Example: `"internalId,scrapeTimestamp"`.

Sample Configuration

{
  "resource": "dataset",
  "operation": "get-items",
  "credentialId": "apify-prod-token",
  "datasetId": "{{ $output.runPollNode.defaultDatasetId }}",
  "offset": "0",
  "limit": "100",
  "fields": "title,url,price,currency,inStock,brand,sku",
  "omit": ""
}

Validation Errors

Error	Cause
`API token and Dataset ID are required`	`credentialId` or `datasetId` is missing or empty.
`Invalid settings`	Internal type mismatch. Verify `resource` is `"dataset"` and `operation` is `"get-items"`.
`APIFY_ERROR` (error port)	Dataset ID not found (may have been deleted or expired), API token lacks access, or a network error occurred.

Output

Success Port

Field	Type	Description
`datasetId`	string	The dataset ID that was queried.
`items`	array	Array of dataset item objects. Each item is a plain object whose fields depend on the actor's output schema, filtered by `fields` / `omit` parameters.
`itemCount`	number	Number of items in this response. May be less than `limit` if the dataset has fewer remaining items after the specified `offset`.
`offset`	number	Offset used for this request, echoed for use in pagination expressions.
`limit`	number	Limit used for this request, echoed for use in pagination expressions.

Error Port

Routes here when the dataset ID is not found, the API token lacks permission to access the dataset, or a network error occurs. Output contains errorCode ("APIFY_ERROR") and message.

Dataset retention: Apify datasets are retained for a limited period (typically 7 days for free accounts, longer for paid plans). If you need to re-process data from old runs, store the raw items in your own database immediately after retrieval — do not rely on Apify's dataset being available indefinitely.

Sample Output

{
  "datasetId": "ds_g1h2i3j4k5l6",
  "items": [
    {
      "title": "Sony WH-1000XM5 Wireless Headphones",
      "url": "https://shop.example.com/sony-wh1000xm5",
      "price": 279.99,
      "currency": "USD",
      "inStock": true,
      "brand": "Sony",
      "sku": "WH1000XM5-BLK"
    },
    {
      "title": "Apple AirPods Pro (2nd Generation)",
      "url": "https://shop.example.com/airpods-pro-2",
      "price": 199.0,
      "currency": "USD",
      "inStock": false,
      "brand": "Apple",
      "sku": "MQTP3LL/A"
    }
  ],
  "itemCount": 2,
  "offset": 0,
  "limit": 100
}

Expression Reference

Expression	Returns
`{{ $output.apify.items }}`	Full array of dataset items for downstream processing.
`{{ $output.apify.itemCount }}`	Item count — compare to `limit` to detect if more pages exist.
`{{ $output.apify.datasetId }}`	Dataset ID echoed for audit or chained calls.
`{{ $output.apify.items[0].title }}`	Title field of the first item.
`{{ $output.apify.items[0].price }}`	Price field of the first item (field name is actor-schema-dependent).
`{{ $output.apify.offset + $output.apify.limit }}`	Next page offset value for pagination loop.

Node Policies & GuardRails

Policy	Recommendation	Severity
API token storage	Use `credentialId`; never embed raw tokens.	Critical
Read after run complete	Only call `dataset/getItems` after confirming the actor run status is `SUCCEEDED`. Reading while the actor is still `RUNNING` returns partial data — items added after your read call are not included.	High
Dataset retention	Persist required items to your own storage immediately after retrieval. Apify datasets expire — data is permanently lost after the retention period without warning.	High
Batch size control	Use `limit` values between 50–500 to balance throughput and memory. Very large limits (5000+) can cause memory pressure in the workflow engine for actors with large, complex item schemas.	High
Field projection	Always use `fields` to return only the columns your downstream mapping needs. Reduces payload size, speeds serialization, and prevents unexpected fields from breaking type-sensitive downstream nodes.	Medium
Pagination completeness	When `itemCount` equals `limit`, assume more items exist. Implement a `Loop` node with offset increment to ensure all items are processed — a single call retrieves at most `limit` items.	Medium

Examples

Read Dataset After Async Actor Run

// Step 1: actor/run → stores runId
// Step 2: Poll loop with actor-run/get until SUCCEEDED
// Step 3: dataset/getItems
{
  "resource": "dataset",
  "operation": "get-items",
  "credentialId": "apify-prod-token",
  "datasetId": "{{ $output.pollNode.defaultDatasetId }}",
  "offset": "0",
  "limit": "200",
  "fields": "name,price,url,inStock"
}

The dataset ID comes from the poll node's output. With 200 items per page and field projection to 4 columns, the payload is compact and loads efficiently into a database upsert node downstream.

Full Paginated Dataset Load via Loop

// Loop node (max 50 iterations, variable: currentOffset starts at 0)
{
  "resource": "dataset",
  "operation": "get-items",
  "credentialId": "apify-prod-token",
  "datasetId": "ds_g1h2i3j4k5l6",
  "offset": "{{ $variables.currentOffset }}",
  "limit": "500",
  "fields": "sku,price,inStock,url"
}
// After each iteration: currentOffset += 500
// Break when itemCount < 500
// Each iteration: upsert items to database

Processes a 15,000-item product dataset in batches of 500. Each batch is upserted into a database table before the loop advances. The loop exits when itemCount drops below 500, indicating the last page.

Selective Field Omission — Privacy Filter

{
  "resource": "dataset",
  "operation": "get-items",
  "credentialId": "apify-prod-token",
  "datasetId": "{{ $input.datasetId }}",
  "offset": "0",
  "limit": "100",
  "fields": "",
  "omit": "email,phone,personalAddress,ipAddress"
}

Returns all actor-defined fields except the four PII fields listed in omit. Use this pattern when an actor returns contact data but the downstream system (e.g. a public analytics dashboard) must not receive personally identifiable information.