Luigi LocalTarget binary file

The problem was with the format of the LocalTarget. Changing it to:

return luigi.LocalTarget('test.npz', format=luigi.format.Nop)

solved the problem. However there was nothing about this in the documentation.

It solved my problem with writing a parquet file in Hadoop. format=luigi.format.Nop did the trick. Thanks!

import luigi
import pandas as pd
import luigi.contrib.hdfs as hdfs

class Hdfs(luigi.Task):
    """
    Writes files into output.
    """
    def __init__(self, *args, **kwargs):
        super(Hdfs, self).__init__( *args, **kwargs)

    def output(self):
        fname_template = f'/data/some_directory/test_luigi.parq'
        return luigi.contrib.hdfs.HdfsTarget(fname_template, format=luigi.format.Nop)

    def run(self):
        with self.output().open('w') as f:
            print(f.path)
            d = pd.DataFrame({'sim_id':[1,2,3]})
            d.to_parquet(f)