Logs

Sources can produce log using console statements, and can throw errors. These logs show up in the sources UI for each source, under LOGS.

Like events, logs can also be consumed programmatically:

SSE

What is SSE?

Server-sent Events (SSE) defines a spec for how servers can send events directly to clients that subscribe to those events, similar to WebSockets and related server-to-client push technologies.

Unlike WebSockets, SSE enables one-way communication from server to clients (WebSockets enable bidirectional communication between server and client, allowing you to pass messages back and forth). If you only need a client to subscribe to events from a server, and don't require bidirectional communication, SSE is simple way to make that happen.

Connecting to the SSE stream directly

Just like for events, logs are published to a source-specific SSE stream.

To connect to this stream, you'll need to:

  • Get the SSE URL for your source using pd list streams, modifying the URL slightly (see below).
  • Connect to the SSE stream, passing your Pipedream API key in the Authorization HTTP header using Bearer Auth.

When you run pd list streams, you'll see output like the following:

λ pd list streams

  NAME                  TYPE   VISIBILITY  SSE
  http                  http   private     https://rt.pipedream.com/sse/dc/dc_abc123/emits

This SSE URL points to the /emits stream, which contains your source's events. Change /emits to /observations to connect to the logs stream.

See this repo for an example Node.js app that processes events from a Pipedream SSE stream.

Most programming languages provide SSE clients that facilitate interaction with SSE streams. For example, the Node.js example repo uses the eventsource npm package, which implements the EventSource API.

pd logs

The pd logs command streams logs for a source directly to your shell:

pd logs <source-id-or-name>

Limits

Pipedream stores the last 100 logs (standard output and standard error) for each source.

Still have questions?

Please reach out if this doc didn't answer your question. We're happy to help!