An treats XCom as a read-only, immutable, single-purpose reference , never as a payload carrier. The word "exclusive" implies three constraints:
Your transactional database will rapidly expand in size.
When using the PythonOperator or TaskFlow API, any value returned by the function is automatically pushed to XCom with the key return_value . 2. Pulling Data airflow xcom exclusive
XComs should pass state and metadata , not data payloads. Pass database IDs, object storage URIs, execution timestamps, or configuration parameters. Never pass raw CSV files, massive logs, or large binaries directly through standard XComs.
Airflow 2.7 introduced an ( BaseXCom ). With it, you can define: An treats XCom as a read-only, immutable, single-purpose
Use pytest-airflow or ruff to scan DAGs for cross-task XCom pulls that don't use key or that pull from non-parent tasks. Example rule: XCOM001 – "Pull from non-upstream task."
Here is an enterprise-grade example of a custom XCom backend using Amazon S3 and pickle serialization (you can substitute pickle with parquet or json depending on security requirements): Never pass raw CSV files, massive logs, or
When a task returns a dict, Airflow pushes each key independently. This can cause fragmentation. Use single return values or multiple_outputs=True carefully.