Actions22
- Cloud Extraction Actions
- Data Actions
- Task Actions
- Task Group Actions
Data → Get Batch Data by Offset
AI-generatedOverview
This node operation retrieves a batch of data rows from Octoparse based on a specified batch ID (lotNo) and offset. It is useful for paginated data extraction scenarios where data is fetched in chunks from a cloud extraction batch. For example, it can be used to process large datasets incrementally by specifying the starting position (offset) and the number of rows (size) to retrieve in each request.
Use Case Examples
- Fetching 100 rows of data starting from offset 0 for batch ID 'batch123' to process in a workflow.
- Retrieving subsequent batches of data by increasing the offset to handle large data extraction results in manageable chunks.
Properties
| Name | Meaning |
|---|---|
| Task Group | Select the task group to find the task in, used to scope the data retrieval. |
| Task | Select the specific task within the task group to retrieve data from. |
| Batch ID | The batch ID (lotNo) returned from starting a cloud extraction, identifying the data batch to retrieve. |
| Offset | Starting position of data retrieval within the batch, used for pagination. |
| Size | Number of rows to retrieve in this batch request, limited between 1 and 1000. |
Output
JSON
data- The retrieved batch data rows from the specified task and batch ID.
Dependencies
- Requires an API key credential for Octoparse API authentication.
Troubleshooting
- Common issues include invalid or expired API credentials causing authentication failures.
- Incorrect batch ID or offset values may result in empty or incomplete data responses.
- API rate limits or network issues may cause request failures; retry logic or error handling is recommended.
Links
- Octoparse API Documentation - Official API documentation for understanding endpoints and parameters used in data retrieval.