1、獲取JSON數據

我們以github上jq項目最新5條評論的JSON數據為例。獲取數據如下:

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5'

結果如下：

[
  {
    "sha": "d25341478381063d1c76e81b3a52e0592a7c997f",
    "commit": {
      "author": {
        "name": "Stephen Dolan",
        "email": "mu@netsoc.tcd.ie",
        "date": "2013-06-22T16:30:59Z"
      },
      "committer": {
        "name": "Stephen Dolan",
        "email": "mu@netsoc.tcd.ie",
        "date": "2013-06-22T16:30:59Z"
      },
      "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161",
      "tree": {
        "sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40",
        "url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40"
      },
      "url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
      "comment_count": 0
    },
    "url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
    "html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f",
    "comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments",
    "author": {
      "login": "stedolan",
    ...

2、高亮並按屬性排序顯示

用 jq '.'即可：

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.'

結果：

[
  {
    "parents": [
      {
        "html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
        "url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
        "sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
      },
      {
        "html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
        "url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
        "sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
      }
    ],
    "committer": {
      "type": "User",
      "received_events_url": "https://api.github.com/users/stedolan/received_events",
      "events_url": "https://api.github.com/users/stedolan/events{/privacy}",
      "repos_url": "https://api.github.com/users/stedolan/repos",
      "organizations_url": "https://api.github.com/users/stedolan/orgs",
...

這里的評論內容比較多，我們現在想拿第一個評論。

3、獲取數組某一項

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.[0]'

結果：

{
  "parents": [
    {
      "html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
      "url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
      "sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
    },
    {
      "html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
      "url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
      "sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
    }
  ],
  "committer": {
    "type": "User",
    "received_events_url": "https://api.github.com/users/stedolan/received_events",
    "events_url": "https://api.github.com/users/stedolan/events{/privacy}",
    "repos_url": "https://api.github.com/users/stedolan/repos",
    "organizations_url": "https://api.github.com/users/stedolan/orgs",
    "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
    "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
    "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
    "login": "stedolan",
    "id": 79765,
    "avatar_url": "https://1.gravatar.com/avatar/31de909d8e55dd07ed782d92ece59842?d=https%3A%2F%2Fidenticons.github.com%2Ffc5b6765b1c9cfaecea48ae71df4d279.png",
    "gravatar_id": "31de909d8e55dd07ed782d92ece59842",
    "url": "https://api.github.com/users/stedolan",
    "html_url": "https://github.com/stedolan",
    "followers_url": "https://api.github.com/users/stedolan/followers",
    "following_url": "https://api.github.com/users/stedolan/following{/other_user}"
  },
  "author": {
    "type": "User",
    "received_events_url": "https://api.github.com/users/stedolan/received_events",
    "events_url": "https://api.github.com/users/stedolan/events{/privacy}",
    "repos_url": "https://api.github.com/users/stedolan/repos",
    "organizations_url": "https://api.github.com/users/stedolan/orgs",
    "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
    "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
    "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
    "login": "stedolan",
    "id": 79765,
    "avatar_url": "https://1.gravatar.com/avatar/31de909d8e55dd07ed782d92ece59842?d=https%3A%2F%2Fidenticons.github.com%2Ffc5b6765b1c9cfaecea48ae71df4d279.png",
    "gravatar_id": "31de909d8e55dd07ed782d92ece59842",
    "url": "https://api.github.com/users/stedolan",
    "html_url": "https://github.com/stedolan",
    "followers_url": "https://api.github.com/users/stedolan/followers",
    "following_url": "https://api.github.com/users/stedolan/following{/other_user}"
  },
  "comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments",
  "html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f",
  "url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
  "commit": {
    "comment_count": 0,
    "url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
    "tree": {
      "url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40",
      "sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40"
    },
    "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161",
    "committer": {
      "date": "2013-06-22T16:30:59Z",
      "email": "mu@netsoc.tcd.ie",
      "name": "Stephen Dolan"
    },
    "author": {
      "date": "2013-06-22T16:30:59Z",
      "email": "mu@netsoc.tcd.ie",
      "name": "Stephen Dolan"
    }
  },
  "sha": "d25341478381063d1c76e81b3a52e0592a7c997f"
}

View Code

可以看到，我們已經拿到了一條完整的評論內容。但我們真正關心的只是評論內容和用戶名，下面來獲取這兩項內容。

4、自定義格式輸出某一項

jq '.[0] | {message: .commit.message, name: .commit.committer.name}'

結果：

{
  "name": "Stephen Dolan",
  "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
}

我們可以看到，已經拿到了想要的內容，並且已經按我們自己定義的格式顯示了。

這里 | 后面的內容是以前面的內容為輸入的，.commit 中的 . 就是指 .[0] 中的內容。

5、自定義格式輸出多項

jq '.[] | {message: .commit.message, name: .commit.committer.name}'

結果：

{
  "name": "Stephen Dolan",
  "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
}
{
  "name": "Stephen Dolan",
  "message": "Reject all overlong UTF8 sequences."
}
{
  "name": "Stephen Dolan",
  "message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
}
{
  "name": "Stephen Dolan",
  "message": "Fix example in manual for `floor`. See #155."
}
{
  "name": "Nicolas Williams",
  "message": "Document floor"
}

這里 .[] 獲取的是數組中的所有項。

我們看到，結果是一個個獨立的JSON對象，如何把結果組合成一個數組呢？

6、以數組形式自定義輸出多項

jq '[.[] | {message: .commit.message, name: .commit.committer.name}]'

結果：

[
  {
    "name": "Stephen Dolan",
    "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
  },
  {
    "name": "Stephen Dolan",
    "message": "Reject all overlong UTF8 sequences."
  },
  {
    "name": "Stephen Dolan",
    "message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
  },
  {
    "name": "Stephen Dolan",
    "message": "Fix example in manual for `floor`. See #155."
  },
  {
    "name": "Nicolas Williams",
    "message": "Document floor"
  }
]

我們可以看到，只要在上一步的命令中內容的兩端加個中括號即可。

最后，我們如果想獲取每個評論的引用評論的url（在parents節點中，有一個或多個）呢？

7、獲取其他內容

jq '[.[] | {message: .commit.message, name: .commit.committer.name, parents: [.parents[].html_url]}]'

結果：

[
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
      "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a"
    ],
    "name": "Stephen Dolan",
    "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
  },
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/ff48bd6ec538b01d1057be8e93b94eef6914e9ef"
    ],
    "name": "Stephen Dolan",
    "message": "Reject all overlong UTF8 sequences."
  },
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7"
    ],
    "name": "Stephen Dolan",
    "message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
  },
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/3dcdc582ea993afea3f5503a78a77675967ecdfa"
    ],
    "name": "Stephen Dolan",
    "message": "Fix example in manual for `floor`. See #155."
  },
  {
    "parents": [
      "https://github.com/stedolan/jq/commit/7c4171d414f647ab08bcd20c76a4d8ed68d9c602"
    ],
    "name": "Nicolas Williams",
    "message": "Document floor"
  }
]

這里用 .parents[].html_url 獲取當前項的 parents 節點中的所有項的 html_url 屬性的內容，然后兩邊加個中括號組裝成數組輸出。

怎么樣，經過這個例子可以看出，用jq處理JSON數據還是很方便強大的吧。

8、注意點

jq能處理的需要是嚴格的JSON格式數據，JSON對象和JSON字符串是不行的，如下面的兩種格式數據jq是不能處理的：

json對象：

{
    a: 1,
    b: {
        c: "abc"
    }
}

json字符串：

'{"a":1,"b":{"c":"abc"}}'

正確的JSON格式：

{
    "a": 1,
    "b": {
        "c": "abc"
    }
}

關於什么是JSON格式數據，請參考：http://www.json.org/

本文原文出自十年后的盧哥哥：http://www.cnblogs.com/lurenjiashuo/，轉載請注明出處。

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 使用jq工具在Shell命令行處理JSON數據 Linux--shell腳本之文本處理工具 shell腳本處理大數據系列之(一)方法小結 Shell 腳本處理用戶輸入 Shell awk文本處理,shell腳本編寫【轉】shell腳本處理字符串的常用方法 Linux Shell 文本處理工具集錦 Linux shell文本處理工具 Linux Shell 文本處理工具 jq處理JSON數據, jq Manual (development version)