Stripe
This page contains the setup guide and reference information for the Stripe source connector.
Prerequisites
- Access to the Stripe account containing the data you wish to replicate
Setup Guide
To authenticate the Stripe connector, you need to use a Stripe API key. Although you may use an existing key, we recommend that you create a new restricted key specifically for Airbyte and grant it Read privileges only. We also recommend granting Read privileges to all available permissions, and configuring the specific data you would like to replicate within Airbyte.
Step 1: Set up Stripe
- Log in to your Stripe account.
- In the top navigation bar, click Developers.
- In the top-left corner, click API keys.
- Click + Create restricted key.
- Choose a Key name, and select Read for all available permissions.
- Click Create key. You may be prompted to enter a confirmation code sent to your email address.
For more information on Stripe API Keys, see the Stripe documentation.
Step 2: Set up the Stripe source connector in Airbyte
-
Log in to your Airbyte Cloud account or your Airbyte Open Source account.
-
In the left navigation bar, click Sources. In the top-right corner, click + New source.
-
Find and select Stripe from the list of available sources.
-
For Source name, enter a name to help you identify this source.
-
For Account ID, enter your Stripe Account ID. This ID begins with
acct_
, and can be found in the top-right corner of your Stripe account settings page. -
For Secret Key, enter the restricted key you created for the connection.
-
For Replication Start Date, use the provided datepicker or enter a UTC date and time programmatically in the format
YYYY-MM-DDTHH:mm:ssZ
. The data added on and after this date will be replicated. -
(Optional) For Lookback Window, you may specify a number of days from the present day to reread data. This allows the connector to retrieve data that might have been updated after its initial creation, and is useful for handling any post-transaction adjustments. This applies only to streams that do not support event-based incremental syncs, please see the list below.
- Leaving the Lookback Window at its default value of 0 means Airbyte will not re-export data after it has been synced.
- Setting the Lookback Window to 1 means Airbyte will re-export data from the past day, capturing any changes made in the last 24 hours.
- Setting the Lookback Window to 7 means Airbyte will re-export and capture any data changes within the last week.
-
(Optional) For Data Request Window, you may specify the time window in days used by the connector when requesting data from the Stripe API. This window defines the span of time covered in each request, with larger values encompassing more days in a single request. Generally speaking, the lack of overhead from making fewer requests means a larger window is faster to sync. However, this also means the state of the sync will persist less frequently. If an issue occurs or the sync is interrupted, a larger window means more data will need to be resynced, potentially causing a delay in the overall process.
For example, if you are replicating three years worth of data:
- A Data Request Window of 365 days means Airbyte makes 3 requests, each for a year. This is generally faster but risks needing to resync up to a year's data if the sync is interrupted.
- A Data Request Window of 30 days means 36 requests, each for a month. This may be slower but minimizes the amount of data that needs to be resynced if an issue occurs.
If you are unsure of which value to use, we recommend leaving this setting at its default value of 365 days.
-
Click Set up source and wait for the tests to complete.
Supported sync modes
The Stripe source connector supports the following sync modes:
- Full Refresh
- Incremental
Supported streams
The Stripe source connector supports the following streams:
- Accounts
- Application Fees (Incremental)
- Application Fee Refunds (Incremental)
- Authorizations (Incremental)
- Balance Transactions (Incremental)
- Bank accounts (Incremental)
- Cardholders (Incremental)
- Cards (Incremental)
- Charges (Incremental)
note
The
amount
column defaults to the smallest currency unit. Check the Stripe docs for more details. - Checkout Sessions (Incremental)
- Checkout Sessions Line Items (Incremental)
- Coupons (Incremental)
- Credit Notes (Incremental)
- Customer Balance Transactions (Incremental)
- Customers (Incremental)
- Disputes (Incremental)
- Early Fraud Warnings (Incremental)
- Events (Incremental)
- External Account Bank Accounts (Incremental)
- External Account Cards (Incremental)
- File Links (Incremental)
- Files (Incremental)
- Invoice Items (Incremental)
- Invoice Line Items
- Invoices (Incremental)
- Payment Intents (Incremental)
- Payment Methods
- Payouts (Incremental)
- Promotion Code (Incremental)
- Persons (Incremental)
- Plans (Incremental)
- Prices (Incremental)
- Products (Incremental)
- Refunds (Incremental)
- Reviews (Incremental)
- Setup Attempts (Incremental)
- Setup Intents (Incremental)
- Shipping Rates (Incremental)
- Subscription Items
- Subscription Schedule (Incremental)
- Subscriptions (Incremental)
- Top Ups (Incremental)
- Transactions (Incremental)
- Transfers (Incremental)
- Transfer Reversals
- Usage Records
Data type mapping
The Stripe API uses the same JSON Schema types that Airbyte uses internally (string
, date-time
, object
, array
, boolean
, integer
, and number
), so no type conversions are performed for the Stripe connector.
Limitations & Troubleshooting
Expand to see details about Stripe connector limitations and troubleshooting.
Connector limitations
Rate limiting
The Stripe connector should not run into Stripe API limitations under normal usage. See Stripe Rate limits documentation. Create an issue if you see any rate limit issues that are not automatically retried successfully.
Stripe API Restriction on Events Data: Access to the events endpoint is guaranteed only for the last 30 days by Stripe. If you use the Full Refresh Overwrite sync, be aware that any events data older than 30 days will be deleted from your target destination and replaced with the data from the last 30 days only. Use an Append sync mode to ensure historical data is retained. Please be aware: this also means that any change older than 30 days will not be replicated using the incremental sync mode. If you want all your synced data to remain up to date, please set up your sync frequency to no more than 30 days.
Troubleshooting
Since the Stripe API does not allow querying objects which were updated since the last sync, the Stripe connector uses the Events API under the hood to implement incremental syncs and export data based on its update date.
However, not all the entities are supported by the Events API, so the Stripe connector uses the created
field or its analogue to query for new data in your Stripe account. These are the entities synced based on the date of creation:
Balance Transactions
Events
File Links
Files
Refunds
Setup Attempts
Shipping Rates
On the other hand, the following streams use the updated
field value as a cursor:
Application Fees
Application Fee Refunds
Authorizations
Bank Accounts
Cardholders
Cards
Charges
Checkout Sessions
Checkout Session Line Items
(cursor field ischeckout_session_updated
)Coupons
Credit Notes
Customer Balance Transactions
Customers
Disputes
Early Fraud Warnings
External Account Bank Accounts
External Account Cards
Invoice Items
Invoices
Payment Intents
Payouts
Promotion Codes
Persons
Plans
Prices
Products
Reviews
Setup Intents
Subscription Schedule
Subscriptions
Top Ups
Transactions
Transfers
Incremental deletes
The Stripe API also provides a way to implement incremental deletes for a limited number of streams:
Bank Accounts
Coupons
Customers
External Account Bank Accounts
External Account Cards
Invoices
Invoice Items
Persons
Plans
Prices
Products
Subscriptions
Each record is marked with is_deleted
flag when the appropriate event happens upstream.
- Check out common troubleshooting issues for the Stripe source connector on our Airbyte Forum.
Data type mapping
Changelog
Version | Date | Pull Request | Subject |
---|---|---|---|
5.2.0 | 2024-01-18 | 34347 | Add new fields invoices and subscription streams. Upgrade the CDK for better memory usage. |
5.1.3 | 2023-12-18 | 33306 | Adding integration tests |
5.1.2 | 2024-01-04 | 33414 | Prepare for airbyte-lib |
5.1.1 | 2024-01-04 | 33926 | Update endpoint for bank_accounts stream |
5.1.0 | 2023-12-11 | 32908 | Read full refresh streams concurrently |
5.0.2 | 2023-12-01 | 33038 | Add stream slice logging for SubStream |
5.0.1 | 2023-11-17 | 32638 | Availability stretegy: check availability of both endpoints (if applicable) - common API + events API |
5.0.0 | 2023-11-16 | 32286 | Fix multiple issues regarding usage of the incremental sync mode for the Refunds , CheckoutSessions , CheckoutSessionsLineItems streams. Fix schemas for the streams: Invoices , Subscriptions , SubscriptionSchedule |
4.5.4 | 2023-11-16 | 32284 | Enable client-side rate limiting |
4.5.3 | 2023-11-14 | 32473 | Have all full_refresh stream syncs be concurrent |
4.5.2 | 2023-11-03 | 32146 | Fix multiple BankAccount issues |
4.5.1 | 2023-11-01 | 32056 | Use CDK version 0.52.8 |
4.5.0 | 2023-10-25 | 31327 | Use concurrent CDK when running in full-refresh |
4.4.2 | 2023-10-24 | 31764 | Base image migration: remove Dockerfile and use the python-connector-base image |
4.4.1 | 2023-10-18 | 31553 | Adjusted Setup Attempts and extended Checkout Sessions stream schemas |
4.4.0 | 2023-10-04 | 31046 | Added margins field to invoice_line_items stream. |
4.3.1 | 2023-09-27 | 30800 | Handle permission issues a non breaking |
4.3.0 | 2023-09-26 | 30752 | Do not sync upcoming invoices, extend stream schemas |
4.2.0 | 2023-09-21 | 30660 | Fix updated state for the incremental syncs |
4.1.1 | 2023-09-15 | 30494 | Fix datatype of invoices.lines property |
4.1.0 | 2023-08-29 | 29950 | Implement incremental deletes, add suggested streams |
4.0.1 | 2023-09-07 | 30254 | Fix cursorless incremental streams |
4.0.0 | 2023-08-15 | 29330 | Implement incremental syncs based on date of update |
3.17.4 | 2023-08-15 | 29425 | Revert 3.17.3 |
3.17.3 | 2023-08-01 | 28911 | Revert 3.17.2 and fix atm_fee property |
3.17.2 | 2023-08-01 | 28911 | Fix stream schemas, remove custom 403 error handling |
3.17.1 | 2023-08-01 | 28887 | Fix Invoices schema |
3.17.0 | 2023-07-28 | 26127 | Add Prices stream |
3.16.0 | 2023-07-27 | 28776 | Add new fields to stream schemas |
3.15.0 | 2023-07-09 | 28709 | Remove duplicate streams |
3.14.0 | 2023-07-09 | 27217 | Add ShippingRates stream |
3.13.0 | 2023-07-18 | 28466 | Pin source API version |
3.12.0 | 2023-05-20 | 26208 | Add new stream Persons |
3.11.0 | 2023-06-26 | 27734 | License Update: Elv2 stream |
3.10.0 | 2023-06-22 | 27132 | Add CreditNotes stream |
3.9.1 | 2023-06-20 | 27522 | Fix formatting |
3.9.0 | 2023-06-19 | 27362 | Add new Streams: Transfer Reversals, Setup Attempts, Usage Records, Transactions |
3.8.0 | 2023-06-12 | 27238 | Add Topups stream; Add Files stream; Add FileLinks stream |
3.7.0 | 2023-06-06 | 27083 | Add new Streams: Authorizations, Cardholders, Cards, Payment Methods, Reviews |
3.6.0 | 2023-05-24 | 25893 | Add ApplicationFeesRefunds stream with parent ApplicationFees |
3.5.0 | 2023-05-20 | 22859 | Add stream Early Fraud Warnings |
3.4.3 | 2023-05-10 | 25965 | Fix Airbyte date-time data-types |
3.4.2 | 2023-05-04 | 25795 | Added CDK TypeTransformer to guarantee declared JSON Schema data-types |
3.4.1 | 2023-04-24 | 23389 | Add customer_tax_ids to Invoices |
3.4.0 | 2023-03-20 | 23963 | Add SetupIntents stream |
3.3.0 | 2023-04-12 | 25136 | Add stream Accounts |
3.2.0 | 2023-04-10 | 23624 | Add new stream Subscription Schedule |
3.1.0 | 2023-03-10 | 19906 | Expand tiers when syncing Plans streams |
3.0.5 | 2023-03-25 | 22866 | Specified date formatting in specification |
3.0.4 | 2023-03-24 | 24471 | Fix stream slices for single sliced streams |
3.0.3 | 2023-03-17 | 24179 | Get customer's attributes safely |
3.0.2 | 2023-03-13 | 24051 | Cache customers stream; Do not request transactions of customers with zero balance. |
3.0.1 | 2023-02-22 | 22898 | Add missing column to Subscriptions stream |
3.0.0 | 2023-02-21 | 23295 | Fix invoice schema |
2.0.0 | 2023-02-14 | 22312 | Another fix of Invoices stream schema + Remove http urls from openapi_spec.json |
1.0.2 | 2023-02-09 | 22659 | Set AvailabilityStrategy for all streams |
1.0.1 | 2023-01-27 | 22042 | Set AvailabilityStrategy for streams explicitly to None |
1.0.0 | 2023-01-25 | 21858 | Update the Subscriptions and Invoices stream schemas |
0.1.40 | 2022-10-20 | 18228 | Update the PaymentIntents stream schema |
0.1.39 | 2022-09-28 | 17304 | Migrate to per-stream states. |
0.1.38 | 2022-09-09 | 16537 | Fix redeem_by field type for customers stream |
0.1.37 | 2022-08-16 | 15686 | Fix the bug when the stream couldn't be fetched due to limited permission set, if so - it should be skipped |
0.1.36 | 2022-08-04 | 15292 | Implement slicing |
0.1.35 | 2022-07-21 | 14924 | Remove additionalProperties field from spec and schema |
0.1.34 | 2022-07-01 | 14357 | Add external account streams - |
0.1.33 | 2022-06-06 | 13449 | Add semi-incremental support for CheckoutSessions and CheckoutSessionsLineItems streams, fixed big in StripeSubStream, added unittests, updated docs |
0.1.32 | 2022-04-30 | 12500 | Improve input configuration copy |
0.1.31 | 2022-04-20 | 12230 | Update connector to use a spec.yaml |
0.1.30 | 2022-03-21 | 11286 | Minor corrections to documentation and connector specification |
0.1.29 | 2022-03-08 | 10359 | Improved performance for streams with substreams: invoice_line_items, subscription_items, bank_accounts |
0.1.28 | 2022-02-08 | 10165 | Improve 404 handling for CheckoutSessionsLineItems stream |
0.1.27 | 2021-12-28 | 9148 | Fix date , arrival\_date fields |
0.1.26 | 2021-12-21 | 8992 | Fix type events.request in schema |
0.1.25 | 2021-11-25 | 8250 | Rearrange setup fields |
0.1.24 | 2021-11-08 | 7729 | Include tax data in checkout_sessions_line_items stream |
0.1.23 | 2021-11-08 | 7729 | Correct payment_intents schema |
0.1.22 | 2021-11-05 | 7345 | Add 3 new streams |
0.1.21 | 2021-10-07 | 6841 | Fix missing start_date argument + update json files for SAT |
0.1.20 | 2021-09-30 | 6017 | Add lookback_window_days parameter |
0.1.19 | 2021-09-27 | 6466 | Use start_date parameter in incremental streams |
0.1.18 | 2021-09-14 | 6004 | Fix coupons and subscriptions stream schemas by removing incorrect timestamp formatting |
0.1.17 | 2021-09-14 | 6004 | Add PaymentIntents stream |
0.1.16 | 2021-07-28 | 4980 | Remove Updated field from schemas |
0.1.15 | 2021-07-21 | 4878 | Fix incorrect percent_off and discounts data filed types |
0.1.14 | 2021-07-09 | 4669 | Subscriptions Stream now returns all kinds of subscriptions (including expired and canceled) |
0.1.13 | 2021-07-03 | 4528 | Remove regex for acc validation |
0.1.12 | 2021-06-08 | 3973 | Add AIRBYTE_ENTRYPOINT for Kubernetes support |
0.1.11 | 2021-05-30 | 3744 | Fix types in schema |
0.1.10 | 2021-05-28 | 3728 | Update data types to be number instead of int |
0.1.9 | 2021-05-13 | 3367 | Add acceptance tests for connected accounts |
0.1.8 | 2021-05-11 | 3566 | Bump CDK connectors |