程序问答   发布时间:2022-06-02  发布网站:大佬教程  code.js-code.com
大佬教程收集整理的这篇文章主要介绍了数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”大佬教程大佬觉得挺不错的,现在分享给大家,也给大家做个参考。

如何解决数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”?

开发过程中遇到数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”的问题如何解决?下面主要结合日常开发的经验,给出你关于数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”的解决方法建议,希望对你解决数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”有所启发或帮助;

您好,我目前正在尝试建立从 @R_502_5561@ Cloud Storage 到 Big query 表的每日数据传输。这些表仅用于存储原始数据(JsON 文件),稍后我会使用计划查询将它们取消嵌套。我手动创建表时没有问题,但我收到错误“读取数据时出错,错误消息:JsON 表遇到太多错误,放弃。行:1;错误:1”当我启动测试传输(甚至如果我之前删除了表中的数据)。

以下是原始数据表/JsON 架构的示例:

{
  "fIElds": [{
    "name": "data","type": "RECORD","mode": "REPEATED","scheR_129_11845@a": {
      "fIElds": [{
        "name": "relationships","mode": "NulLABLE","scheR_129_11845@a": {
          "fIElds": [{
            "name": "createdby","scheR_129_11845@a": {
              "fIElds": [{
                "name": "data","scheR_129_11845@a": {
                  "fIElds": [{
                    "name": "type","type": "StriNG","mode": "NulLABLE"
                  },{
                    "name": "ID","type": "IntegeR","mode": "NulLABLE"
                  }]
                }
              }]
            }
          },{
            "name": "valIDationWorkflow",{
            "name": "projects",{
            "name": "expensesReport",{
            "name": "agency",{
            "name": "files","mode": "REPEATED"
              }]
            }
          },{
            "name": "resource",{
            "name": "valIDations",{
            "name": "orders","mode": "NulLABLE"
                  }]
                }
              }]
            }
          }]
        }
      },{
        "name": "attributes","scheR_129_11845@a": {
          "fIElds": [{
            "name": "plAnnedTimes","mode": "REPEATED"
          },{
            "name": "state","mode": "NulLABLE"
          },{
            "name": "exceptionalTimes",{
            "name": "closed","type": "BooleAN",{
            "name": "informationComments",{
            "name": "regularTimes","scheR_129_11845@a": {
              "fIElds": [{
                "name": "batch","scheR_129_11845@a": {
                  "fIElds": [{
                    "name": "title","mode": "NulLABLE"
                  }]
                }
              },{
                "name": "workUnitType","scheR_129_11845@a": {
                  "fIElds": [{
                    "name": "name",{
                    "name": "activityType",{
                    "name": "reference",{
                "name": "project","scheR_129_11845@a": {
                  "fIElds": [{
                    "name": "reference",{
                "name": "row","mode": "NulLABLE"
              },{
                "name": "delivery","scheR_129_11845@a": {
                  "fIElds": [{
                    "name": "endDate","type": "DATE",{
                    "name": "startDate",{
                    "name": "title",{
                "name": "startDate",{
                "name": "duration","type": "float",{
                "name": "ID","mode": "NulLABLE"
              }]
            }
          },{
            "name": "updateDate","type": "timestAMP",{
            "name": "creationDate",{
            "name": "absencesTimes","scheR_129_11845@a": {
              "fIElds": [{
                "name": "workUnitType",{
            "name": "term","mode": "NulLABLE"
          }]
        }
      },{
        "name": "type","mode": "NulLABLE"
      },{
        "name": "ID","mode": "NulLABLE"
      }]
    }
  },{
    "name": "date","mode": "NulLABLE"
  }]
}

我知道 BQ 使用 JsON 换行符分隔格式效果更好,但该表必须包含原始数据,即使它最后只有 1 行。最奇怪的是,传输适用于一些具有相似架构的文件。

我应该怎么做才能使这些转移工作?

感谢您的帮助

解决方法

关于 GCP 文档,我会说 using JSON newline delimited format is a known limitation。

"JSON data must be newline delimited. Each JSON object must be on a separate line in the file."

大佬总结

以上是大佬教程为你收集整理的数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”全部内容,希望文章能够帮你解决数据传输到 Big Query 错误“读取数据时出错,错误消息:JSON 表遇到太多错误,放弃。行:1;错误:1”所遇到的程序开发问题。

如果觉得大佬教程网站内容还不错,欢迎将大佬教程推荐给程序员好友。

本图文内容来源于网友网络收集整理提供,作为学习参考使用,版权属于原作者。
如您有任何意见或建议可联系处理。小编QQ:384754419,请注明来意。