TransferQueue

Defined in: src/transfers/TransferQueue.ts:139

Minimal transfer queue with concurrency, pause/resume, cancellation, and drain summaries.

Wrap a TransferEngine with a queue when you need to run many transfers concurrently with bounded parallelism, observe per-job progress, or drive a UI from a single source of truth. Items are FIFO; failures and successes are surfaced via observers and in the final TransferQueueSummary.

Example

import {
  TransferQueue,
  createProviderTransferExecutor,
} from "@zero-transfer/sdk";

const queue = new TransferQueue({
  concurrency: 4,
  executor: createProviderTransferExecutor({ client }),
  onProgress: (e) => console.log(`${e.jobId}: ${e.bytesTransferred}`),
  onError: (item, err) => console.error(`${item.job.id} failed`, err),
});

for (const file of files) {
  queue.enqueue({
    id: file.name,
    operation: "upload",
    source: { profile: localProfile, path: file.path },
    destination: { profile: s3Profile, path: `/lake/${file.name}` },
  });
}

const summary = await queue.drain();
console.log(`Completed ${summary.completed} / ${summary.total}`);

Constructors

Constructor

new TransferQueue(options?: TransferQueueOptions): TransferQueue;

Defined in: src/transfers/TransferQueue.ts:158

Creates a transfer queue.

Parameters

Parameter	Type	Description
options	TransferQueueOptions	Queue engine, concurrency, executor, and observer options.

Returns

TransferQueue

Methods

add()

add(job: TransferJob, executor?: TransferExecutor): TransferQueueItem;

Defined in: src/transfers/TransferQueue.ts:172

Adds a transfer job to the queue.

Parameters

Parameter	Type
job	TransferJob
executor?	TransferExecutor

Returns

TransferQueueItem

---

cancel()

cancel(jobId: string): boolean;

Defined in: src/transfers/TransferQueue.ts:212

Cancels a queued or running job.

Parameters

Parameter	Type
jobId	string

Returns

boolean

---

get()

get(jobId: string):
  | TransferQueueItem
  | undefined;

Defined in: src/transfers/TransferQueue.ts:234

Returns a queued item snapshot by id.

Parameters

Parameter	Type
jobId	string

Returns

| TransferQueueItem | undefined

---

list()

list(): TransferQueueItem[];

Defined in: src/transfers/TransferQueue.ts:240

Lists queue item snapshots in insertion order.

Returns

TransferQueueItem[]

---

pause()

pause(): void;

Defined in: src/transfers/TransferQueue.ts:197

Pauses dispatch of new queued jobs. Running jobs are allowed to finish.

Returns

void

---

resume()

resume(): void;

Defined in: src/transfers/TransferQueue.ts:202

Resumes dispatch of queued jobs on the next run() call.

Returns

void

---

run()

run(options?: TransferQueueRunOptions): Promise<TransferQueueSummary>;

Defined in: src/transfers/TransferQueue.ts:245

Drains currently queued jobs until complete, failed, canceled, or paused.

Parameters

Parameter	Type
options	TransferQueueRunOptions

Returns

Promise<TransferQueueSummary>

---

setConcurrency()

setConcurrency(concurrency: number): void;

Defined in: src/transfers/TransferQueue.ts:207

Updates queue concurrency for subsequent drains.

Parameters

Parameter	Type
concurrency	number

Returns

void

---

summarize()

summarize(): TransferQueueSummary;

Defined in: src/transfers/TransferQueue.ts:254

Returns a queue summary without executing more work.

Returns

TransferQueueSummary