[
https://issues.apache.org/jira/browse/AIRFLOW-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aizhamal Nurmamat kyzy updated AIRFLOW-1347:
--------------------------------------------
Component/s: (was: db)
(was: DagRun)
database
> sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL
> for primary key
> ------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-1347
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1347
> Project: Apache Airflow
> Issue Type: Bug
> Components: database
> Affects Versions: 1.8.0
> Environment: ubuntu 12.04
> mysql 5.7.18
> Reporter: Antoine Grouazel
> Priority: Major
> Attachments: bug_airflow.txt
>
>
> *Description of the issue*: the scheduler cannot update the mysql db via
> SqlAlchemy and the tasks are queued and not rescheduled. It affects all our
> DAGs but not all the tasks nor all the time. Manually clear a task having
> such issue can fix the problem for the given task instance.
> There is a kind of random behavior...
> Upgrading mysql 5.7.17 to 5.7.18 does not fix the issue. (x)
> Issue is neitheir fixed in the lastest version 1.8.2rc1 . (x)
> Don't know if there is a link with
> https://issues.apache.org/jira/browse/AIRFLOW-977. (?)
> {quote}
> {{[2017-06-24 00:31:10,272] {models.py:167} INFO - Filling up the DagBag from
> /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py
> [2017-06-24 00:31:10,427] {base_task_runner.py:112} INFO - Running: ['bash',
> '-c', u'airflow run Sentinel1_copernicus_download latest_only
> 2017-06-23T00:31:00 --job_id 293 --raw -sd
> DAGS_FOLDER/sentinel1_copernicus_download.py']
> [2017-06-24 00:31:11,449] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:11,449] {__init__.py:57} INFO - Using executor
> CeleryExecutor
> [2017-06-24 00:31:11,668] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:11,668] {driver.py:120} INFO - Generating grammar tables
> from /usr/lib/python2.7/lib2to3/Grammar.txt
> [2017-06-24 00:31:11,724] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:11,723] {driver.py:120} INFO - Generating grammar tables
> from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
> [2017-06-24 00:31:12,067] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,066] {models.py:167} INFO - Filling up the DagBag from
> /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py
> [2017-06-24 00:31:12,184] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,184] {models.py:1126} INFO - Dependencies all met for
> <TaskInstance: Sentinel1_copernicus_download.latest_only 2017-06-23 00:31:00
> [queued]>
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,188] {models.py:1126} INFO - Dependencies all met for
> <TaskInstance: Sentinel1_copernicus_download.latest_only 2017-06-23 00:31:00
> [queued]>
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,188] {models.py:1318} INFO -
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask:
> --------------------------------------------------------------------------------
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: Starting
> attempt 1 of 3
> [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask:
> --------------------------------------------------------------------------------
> [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: Traceback
> (most recent call last):
> [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/bin/airflow", line 28, in <module>
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask:
> args.func(args)
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 422, in run
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask:
> pool=args.pool,
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 53, in
> wrapper
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: result
> = func(*args, **kwargs)
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1328, in run
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask:
> session.commit()
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 801,
> in commit
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask:
> self.transaction.commit()
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 392,
> in commit
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask:
> self._prepare_impl()
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 372,
> in _prepare_impl
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask:
> self.session.flush()
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line
> 2019, in flush
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask:
> self._flush(objects)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line
> 2137, in _flush
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask:
> transaction.rollback(_capture_exception=True)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line
> 60, in __exit__
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask:
> compat.reraise(exc_type, exc_value, exc_tb)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line
> 2101, in _flush
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask:
> flush_context.execute()
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line
> 373, in execute
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask:
> rec.execute(self)
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line
> 532, in execute
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: uow
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line
> 170, in save_obj
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask:
> mapper, table, update)
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line
> 634, in _emit_update_statements
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: lambda
> rec: (
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line
> 531, in _collect_update_commands
> [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask: "key
> value on column %s" % (table, col))
> [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask:
> sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL
> for primary key value on column task_instance.execution_date
> [2017-06-24 00:31:12,311] {jobs.py:2083} INFO - Task exited with return code
> 1}}{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
