Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: [Athena] does not list tables #10765

Closed
1 task done
anjakefala opened this issue Feb 1, 2025 · 3 comments
Closed
1 task done

bug: [Athena] does not list tables #10765

anjakefala opened this issue Feb 1, 2025 · 3 comments
Labels
bug Incorrect behavior inside of ibis

Comments

@anjakefala
Copy link
Contributor

What happened?

Image

When I try to list the tables in a database, the result is empty.

What version of ibis are you using?

main branch commit 3d10def

What backend(s) are you using, if any?

Athena

Relevant log output

In [6]: con.drop_database('mydatabase', force=True)
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
Cell In[6], line 1
----> 1 con.drop_database('mydatabase', force=True)

File ~/git/ibis/ibis/backends/athena/__init__.py:461, in Backend.drop_database(self, name, catalog, force)
    459 name = sg.table(name, catalog=catalog, quoted=self.compiler.quoted)
    460 sql = sge.Drop(this=name, kind="SCHEMA", replace=force)
--> 461 with self._safe_raw_sql(sql, unload=False):
    462     pass

File ~/anaconda3/envs/ibis-dev/lib/python3.11/contextlib.py:137, in _GeneratorContextManager.__enter__(self)
    135 del self.args, self.kwds, self.func
    136 try:
--> 137     return next(self.gen)
    138 except StopIteration:
    139     raise RuntimeError("generator didn't yield") from None

File ~/git/ibis/ibis/backends/athena/__init__.py:291, in Backend._safe_raw_sql(self, query, unload, *args, **kwargs)
    289     query = query.sql(self.dialect)
    290 with self.con.cursor(unload=unload) as cur:
--> 291     yield cur.execute(query, *args, **kwargs)

File ~/anaconda3/envs/ibis-dev/lib/python3.11/site-packages/pyathena/arrow/cursor.py:148, in ArrowCursor.execute(self, operation, parameters, work_group, s3_staging_dir, cache_size, cache_expiration_time, result_reuse_enable, result_reuse_minutes, paramstyle, **kwargs)
    137     self.result_set = AthenaArrowResultSet(
    138         connection=self._connection,
    139         converter=self._converter,
   (...)
    145         **kwargs,
    146     )
    147 else:
--> 148     raise OperationalError(query_execution.state_change_reason)
    149 return self

OperationalError: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to drop database: InvalidOperationException(message:Database mydatabase is not empty.))

In [7]: con.list_tables('mydatabase')
Out[7]: []

Code of Conduct

  • I agree to follow this project's Code of Conduct
@anjakefala anjakefala added the bug Incorrect behavior inside of ibis label Feb 1, 2025
@cpcloud
Copy link
Member

cpcloud commented Feb 1, 2025

This is expected behavior: the first argument to list_tables is like, not database:

    def list_tables(
        self, like: str | None = None, database: tuple[str, str] | str | None = None
    ) -> list[str]:

@cpcloud cpcloud closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2025
@github-project-automation github-project-automation bot moved this from backlog to done in Ibis planning and roadmap Feb 1, 2025
@cpcloud
Copy link
Member

cpcloud commented Feb 1, 2025

FYI in 10.0, these arguments will be keyword-only, so it's a bit less error prone.

You'll no longer be able to write con.list_tables without specifying either like=... or database=... or both, all as keyword arguments.

@anjakefala
Copy link
Contributor Author

Oh, sorry for mixing it up, but that's a helpful API change! Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis
Projects
Status: done
Development

No branches or pull requests

2 participants