Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gokart.build hides Exceptions occured in run() #209

Closed
Hi-king opened this issue Jun 8, 2021 · 5 comments
Closed

gokart.build hides Exceptions occured in run() #209

Hi-king opened this issue Jun 8, 2021 · 5 comments

Comments

@Hi-king
Copy link
Member

Hi-king commented Jun 8, 2021

import gokart
class A(gokart.TaskOnKart):
    def run(self):
        raise Exception()
        self.dump("done")
gokart.build(A())

FileNotFoundError: [Errno 2] No such file or directory: './resources/main/A_c72fa9f02fa5c6f019adbf9fe425c9eb.pkl'

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-34-6bff8168acc9> in <module>()
      5         raise Exception()
      6         self.dump("done")
----> 7 gokart.build(A())

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/gokart/build.py in build(task, verbose, return_value, reset_register)
     49     with HideLogger(verbose):
     50         luigi.build([task], local_scheduler=True)
---> 51     return _get_output(task) if return_value else None

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/gokart/build.py in _get_output(task)
     31     if type(output) == list:
     32         return [x.load() for x in output]
---> 33     return output.load()
     34 
     35 

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/gokart/target.py in load(self)
     26 
     27     def load(self) -> Any:
---> 28         return self.wrap_with_lock(self._load)()
     29 
     30     def dump(self, obj, lock_at_dump: bool = True) -> None:

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/gokart/target.py in _load(self)
     93 
     94     def _load(self) -> Any:
---> 95         with self._target.open('r') as f:
     96             return self._processor.load(f)
     97 

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/luigi/local_target.py in open(self, mode)
    163 
    164         elif rwmode == 'r':
--> 165             fileobj = FileWrapper(io.BufferedReader(io.FileIO(self.path, mode)))
    166             return self.format.pipe_reader(fileobj)
    167 

FileNotFoundError: [Errno 2] No such file or directory: './resources/__main__/A_c72fa9f02fa5c6f019adbf9fe425c9eb.pkl'

@vaaaaanquish
Copy link
Contributor

Everytime I'm debugging it as follows.

gokart.build(A(), verbose=True)

I've always wanted a better way.

@Hi-king
Copy link
Member Author

Hi-king commented Jun 8, 2021

Thx.
I think it can be better to output task logs when final output will not be found :)

@TaylerUva
Copy link
Contributor

My work arround which prints the raised exception

import gokart
class A(gokart.TaskOnKart):
    def on_failure(self, exception):
        import traceback
        traceback.print_exc()
        return super().on_failure(exception)
        
    def run(self):
        raise Exception()
        self.dump("done")
gokart.build(A())

@vaaaaanquish
Copy link
Contributor

vaaaaanquish commented Aug 7, 2021

It's similar: #224

I think It is best if luigi returns stacktrace. ( spotify/luigi#3095 )
If not it would be good to implement TaylerUva's method in gokart

@Hi-king
Copy link
Member Author

Hi-king commented Sep 7, 2021

#237 lessened the pain. For now, gokart.build(Task(), log_level=logging.ERROR) let me know Exceptions() in run() :)

class Task(gokart.TaskOnKart):
    def run(self):
        raise Exception()

gokart.build(Task(), log_level=logging.ERROR)

Thx all!

@Hi-king Hi-king closed this as completed Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants