Drop Airflow 2 Support in Edge Provider (apache#59143)

* Drop Airflow 2 Support in Edge Provider

* Fix pytests

* Add note about support in changelog

Author: jscheffl

.pre-commit-config.yaml

@@ -268,7 +268,8 @@ repos:
           ^.*openapi.*\.yaml$|
           ^\.pre-commit-config\.yaml$|
           ^.*reproducible_build\.yaml$|
-          ^.*pnpm-lock\.yaml$
+          ^.*pnpm-lock\.yaml$|
+          ^.*-generated\.yaml$
   - repo: https://github.com/ikamensh/flynt
     rev: 97be693bf18bc2f050667dd282d243e2824b81e2  # frozen: 1.0.6
     hooks:

.rat-excludes

@@ -313,7 +313,7 @@ www-hash.txt
 *generated.*
 /src/airflow/providers/keycloak/auth_manager/openapi/v2-keycloak-auth-manager-generated.yaml
 /src/airflow/providers/edge3/plugins/www/*
-/src/airflow/providers/edge3/openapi/v2-edge-generated.yaml
+/src/airflow/providers/edge3/worker_api/v2-edge-generated.yaml
 /src/airflow/providers/fab/auth_manager/api_fastapi/openapi/v2-fab-auth-manager-generated.yaml
 /src/airflow/providers/fab/www/static/dist/*
 /any/dag_id=dag_for_testing_redis_task_handler/run_id=test/task_id=task_for_testing_redis_log_handler/attempt=1.log

dev/breeze/src/airflow_breeze/global_constants.py

@@ -736,7 +736,7 @@ def get_airflow_extras():
     {
         "python-version": "3.10",
         "airflow-version": "2.11.0",
-        "remove-providers": "common.messaging fab git keycloak",
+        "remove-providers": "common.messaging edge3 fab git keycloak",
         "run-unit-tests": "true",
     },
     {

go-sdk/pkg/edgeapi/client.go

@@ -27,7 +27,7 @@ import (
 
 //go:generate -command openapi-gen go run github.com/ashb/oapi-resty-codegen@latest --config oapi-codegen.yml
 
-//go:generate openapi-gen https://raw.githubusercontent.com/apache/airflow/refs/tags/providers-edge3/1.3.0/providers/edge3/src/airflow/providers/edge3/openapi/v2-edge-generated.yaml
+//go:generate openapi-gen https://raw.githubusercontent.com/apache/airflow/refs/tags/providers-edge3/1.3.0/providers/edge3/src/airflow/providers/edge3/worker_api/v2-edge-generated.yaml
 
 func WithEdgeAPIJWTKey(key []byte) ClientOption {
 	return func(c *Client) error {

providers/edge3/.pre-commit-config.yaml

@@ -37,7 +37,7 @@ repos:
         files: |
           (?x)
           ^src/airflow/providers/edge3/plugins/www/.*\.(js|ts|tsx|yaml|css|json)$|
-          ^src/airflow/providers/edge3/openapi/v2-edge-generated.yaml$
+          ^src/airflow/providers/edge3/worker_api/v2-edge-generated.yaml$
         exclude: |
           (?x)
           ^src/airflow/providers/edge3/plugins/www/node-modules/.*|

providers/edge3/README.rst

@@ -65,7 +65,7 @@ Requirements
 ==========================================  ====================
 PIP package                                 Version required
 ==========================================  ====================
-``apache-airflow``                          ``>=2.11.0,!=3.1.0``
+``apache-airflow``                          ``>=3.0.0,!=3.1.0``
 ``apache-airflow-providers-common-compat``  ``>=1.10.0``
 ``pydantic``                                ``>=2.11.0``
 ``retryhttp``                               ``>=1.2.0,!=1.3.0``

providers/edge3/docs/architecture.rst

@@ -63,7 +63,7 @@ deployed outside of the central Airflow cluster is connected via HTTP(s) to the
 * **Workers** (Central) - Execute the assigned tasks - most standard setup has local or centralized workers, e.g. via Celery
 * **Edge Workers** - Special workers which pull tasks via HTTP(s) as provided as feature via this provider package
 * **Scheduler** - Responsible for adding the necessary tasks to the queue. The EdgeExecutor is running as a module inside the scheduler.
-* **API server** (webserver in Airflow 2.x) - HTTP REST API Server provides access to Dag/task status information. The required end-points are
+* **API server** - HTTP REST API Server provides access to Dag/task status information. The required end-points are
   provided by the Edge provider plugin. The Edge Worker uses this API to pull tasks and send back the results.
 * **Database** - Contains information about the status of tasks, Dags, Variables, connections, etc.
 
@@ -77,7 +77,7 @@ In detail the parts of the Edge provider are deployed as follows:
   need to set the ``executor`` configuration option in the ``airflow.cfg`` file to
   ``airflow.providers.edge3.executors.EdgeExecutor``. For more details see :doc:`edge_executor`. Note that also
   multiple executors can be used in parallel together with the EdgeExecutor.
-* **API server** (webserver in Airflow 2.x) - The API server is providing REST endpoints to the web UI as well
+* **API server** - The API server is providing REST endpoints to the web UI as well
   as serves static files. The Edge provider adds a plugin that provides additional REST API for the Edge Worker
   as well as UI elements to manage workers (not available in Airflow 3.0).
   The API server is responsible for handling requests from the Edge Worker and sending back the results. To

providers/edge3/docs/changelog.rst

@@ -27,6 +27,14 @@
 Changelog
 ---------
 
+.. warning::
+    This release of the Edge3 provider drops support for Airflow versions below 3.0.0.
+
+    The support for Airflow 2.10-2.11 was experimental and GA for the provider is only for Airflow 3.0+.
+    Productive operation was not intended in Airflow 2.x, therefore the support for Airflow 2.x is now dropped
+    earlier than the usual release support policy would indicate.
+
+
 1.6.0
 .....
 

providers/edge3/docs/deployment.rst

@@ -113,7 +113,7 @@ process as and wait until all running tasks are completed. Also in a console you
 If you want to monitor the remote activity and worker, use the UI plugin which
 is included in the provider package and install it on the api-server / webserver and use the
 "Admin" - "Edge Worker Hosts" and "Edge Worker Jobs" pages.
-(Note: The plugin is not available on Airflow 3.0 UI, it is only in 2.x and 3.1++)
+(Note: The plugin is not available on Airflow 3.0 UI, it is only in 3.1++)
 
 If you want to check status of the worker via CLI you can use the command
 
@@ -150,7 +150,7 @@ Worker status can be checked via the web UI in the "Admin" - "Edge Worker" page.
 
 .. note::
 
-    Airflow 3.0 does not support UI plugins. The UI plugin is only available in Airflow 2.10 and in 3.1 and newer.
+    Airflow 3.0 does not support UI plugins. The UI plugin is only available in Airflow 3.1 and newer.
     Alternatively you can use the CLI commands as described in :ref:`deployment:maintenance-mgmt-cli`.
 
 

providers/edge3/docs/index.rst

@@ -108,12 +108,12 @@ For the minimum Airflow version supported, see ``Requirements`` below.
 Requirements
 ------------
 
-The minimum Apache Airflow version supported by this provider distribution is ``2.11.0``.
+The minimum Apache Airflow version supported by this provider distribution is ``3.0.0``.
 
 ==========================================  ====================
 PIP package                                 Version required
 ==========================================  ====================
-``apache-airflow``                          ``>=2.11.0,!=3.1.0``
+``apache-airflow``                          ``>=3.0.0,!=3.1.0``
 ``apache-airflow-providers-common-compat``  ``>=1.10.0``
 ``pydantic``                                ``>=2.11.0``
 ``retryhttp``                               ``>=1.2.0,!=1.3.0``