Installing the Airflow MySQL provider fails on pkg-config or mysqlclient
pip install apache-airflow-providers-mysql stops with a pkg-config errorShort answer
The Airflow MySQL provider depends on mysqlclient, which pip compiles against the MySQL C client library. The build asks pkg-config for the compiler flags. If pkg-config or the development package is missing, the build stops. Install pkg-config, build tools and the client development package, then rerun pip.
Fix it fast
1Find out which cause you have
Scroll up in the pip output to the first error line from the mysqlclient build, then check that pkg-config exists and knows the MySQL client.
which pkg-config pkg-config --cflags --libs mysqlclient2Match what you see, then jump to its fix
What does the error look like?
pip install apache-airflow-providers-mysql stops with a pkg-config errorBuilding the mysqlclient wheel fails during the install
Cause 1 of 2
pkg-config, a compiler or the client headers are not installed
1Check
The provider lists mysqlclient as a requirement, and mysqlclient gets its flags by running pkg-config for the mysqlclient library. Without pkg-config, a compiler and the client development files, there is nothing to build against.
which pkg-config gcc
dpkg -s default-libmysqlclient-dev2Fix
Install the packages the mysqlclient project lists for Debian and Ubuntu, then install the provider again.
sudo apt-get install python3-dev default-libmysqlclient-dev build-essential pkg-config
pip install apache-airflow-providers-mysql3Verify
pkg-config now answers, and the provider imports.
pkg-config --cflags --libs mysqlclient
python -c "import MySQLdb; print('ok')"Confirmed by github.com
Cause 2 of 2
pkg-config is installed but does not return flags for mysqlclient
1Check
mysqlclient reads its flags from pkg-config by default, but it also accepts them from two environment variables. If pkg-config cannot report them for your client library, you can pass the flags yourself.
pkg-config mysqlclient --cflags --libs2Fix
Set MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS from whatever pkg-config reports for your client library (or write the paths by hand), then install again in the same shell.
export MYSQLCLIENT_CFLAGS=`pkg-config mysqlclient --cflags`
export MYSQLCLIENT_LDFLAGS=`pkg-config mysqlclient --libs`
pip install mysqlclient3Verify
The wheel builds, and the provider installs.
pip install apache-airflow-providers-mysql
python -c "import MySQLdb; print('ok')"Confirmed by github.com
How do I stop it coming back?
- Install the system packages before pip install on every new server or image, and keep them in your Dockerfile or provisioning script.
- Pin the provider version in your requirements file so a rebuild installs the same mysqlclient requirement.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- ShopGoAn e-commerce aggregator with a Python scraping engine behind its catalog, past 10K Google Play downloads.
- TaxiDexMy own multi-tenant taxi dispatch SaaS: API, dispatcher portal, driver and passenger apps, and the servers underneath.
- QistLockA NestJS and React SaaS driving an Android Device Owner agent that locks financed phones when payments lapse.
Related fixes
Databases
pip install mysqlclient fails to build on Windows, Linux or macOS
error: metadata-generation-failed
mysqlclient is a C extension.
3 causes, each with a fixDatabases
pip cannot build the mariadb package because mariadb_config is missing
Please make sure, that MariaDB Connector/C is installed on your system.
The mariadb Python package builds against MariaDB Connector/C, and it finds the library through the mariadb_config program.
3 causes, each with a fixDatabases
Compiling a C or C++ program that includes mysql.h fails
undefined reference to `mysql_init'
The compiler only finds mysql.h when the MySQL client development files are installed and their folder is on the include path.
3 causes, each with a fixDatabases
Django cannot load MySQLdb and asks whether mysqlclient is installed
Error loading MySQLdb module: Did you install mysqlclient or MySQL-python?
Django's MySQL backend imports MySQLdb, which is provided by the mysqlclient package.
3 causes, each with a fixDatabases
python manage.py migrate finishes but the tables for my app are missing
relation "myapp_book" does not exist
migrate only applies migration files that exist.
3 causes, each with a fixDatabases
An EC2 instance cannot reach its RDS database
Could not open connection to the host, on port <port>: Connect failed
RDS denies all access until a security group allows it.
3 causes, each with a fix