大佬教程收集整理的这篇文章主要介绍了Dask:不能安全地将非等效的 float64 转换为 int64,但只有 object 和 int64 列,大佬教程大佬觉得挺不错的,现在分享给大家,也给大家做个参考。
我尝试阅读和处理 60 列 (150MB) 的 CSV 文件:
df = dataframe.read_csv(
path,**dialect,dtype=dtype,enCoding=enCoding,storage_options=storage_options,)
但是当我尝试 len(df)
时,它会引发 TypeError: cAnnot safely cast non-equivalent float64 to int64
但是,在 dtype
中,我只为我的列定义指定了 Int64
和 object
类型。
这个浮点转换从何而来?
编辑:经过调查,问题与定义为 int 的列有关,其中包含(在 CSV 中很远)一些字符串值。错误显然不明确
编辑:堆栈跟踪
TraceBACk (most recent call last):
file "/home/benjamin/try/try-BACk/apps/data/tasks.py",line 168,in csv_to_parquet_task
df.to_parquet(
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/dataframe/core.py",line 3974,in to_parquet
return to_parquet(self,path,*args,**kwargs)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/dataframe/io/parquet/core.py",line 508,in to_parquet
out = out.compute(**compute_kwargs)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/base.py",line 167,in compute
(result,) = compute(self,traverse=false,**kwargs)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/base.py",line 452,in compute
results = schedule(dsk,keys,**kwargs)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/threaded.py",line 76,in get
results = get_async(
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/local.py",line 486,in get_async
raise_exception(exc,tb)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/local.py",line 316,in reraise
raise exc
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/local.py",line 222,in execute_task
result = _execute_task(task,data)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/core.py",line 121,in _execute_task
return func(*(_execute_task(a,cachE) for a in args))
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/core.py",in <genexpr>
return func(*(_execute_task(a,cachE) for a in args))
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/dask/dataframe/io/csv.py",line 150,in pandas_read_text
df = reader(bio,**kwargs)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/io/parsers.py",line 688,in read_csv
return _read(filepath_or_buffer,kwds)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/io/parsers.py",line 460,in _read
data = parser.read(nrows)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/io/parsers.py",line 1198,in read
ret = self._@R_489_10846@ne.read(nrows)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/io/parsers.py",line 2157,in read
data = self._reader.read(nrows)
file "pandas/_libs/parsers.pyx",line 847,in pandas._libs.parsers.TextReader.read
file "pandas/_libs/parsers.pyx",line 862,in pandas._libs.parsers.TextReader._read_low_memory
file "pandas/_libs/parsers.pyx",line 941,in pandas._libs.parsers.TextReader._read_rows
file "pandas/_libs/parsers.pyx",line 1073,in pandas._libs.parsers.TextReader._convert_column_data
file "pandas/_libs/parsers.pyx",line 1104,in pandas._libs.parsers.TextReader._convert_tokens
file "pandas/_libs/parsers.pyx",line 1182,in pandas._libs.parsers.TextReader._convert_with_dtype
File "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/core/arrays/Integer.py",line 385,in _from_sequence_of_Strings
return cls._from_sequence(scalars,dtype,copy)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/core/arrays/Integer.py",line 378,in _from_sequence
return Integer_array(scalars,copy=copy)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/core/arrays/Integer.py",line 160,in Integer_array
values,mask = coerce_to_array(values,line 275,in coerce_to_array
values = safe_cast(values,copy=falsE)
file "/home/benjamin/.cache/pypoetry/virtualenvs/try-dKznE2K2-py3.8/lib/python3.8/site-packages/pandas/core/arrays/Integer.py",line 179,in safe_cast
raise TypeError(
TypeError: cAnnot safely cast non-equivalent float64 to int64
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)
以上是大佬教程为你收集整理的Dask:不能安全地将非等效的 float64 转换为 int64,但只有 object 和 int64 列全部内容,希望文章能够帮你解决Dask:不能安全地将非等效的 float64 转换为 int64,但只有 object 和 int64 列所遇到的程序开发问题。
如果觉得大佬教程网站内容还不错,欢迎将大佬教程推荐给程序员好友。
本图文内容来源于网友网络收集整理提供,作为学习参考使用,版权属于原作者。
如您有任何意见或建议可联系处理。小编QQ:384754419,请注明来意。