文件

mc batch generate

在版本 MinIO 中變更:RELEASE.2022-10-08T20-11-00Z 或更新版本

語法

mc batch generate 命令會為指定的任務類型建立基本的 YAML 格式範本檔案。

MinIO 建立檔案後,請在您偏好的文字編輯器工具中開啟它以進一步自訂。您可以在每個批次檔案中定義一個任務定義。

請參閱 任務類型 以取得您可以產生的支援任務。

以下命令會為 myminio 別名中的 mybucket 儲存貯體建立複寫任務的基本 YAML 檔案。

mc batch generate myminio replicate

命令的語法如下

mc [GLOBALFLAGS] batch generate \
                       ALIAS   \
                       JOBTYPE
  • 括號 [] 表示選用參數。

  • 同一行的參數是相互依存的。

  • 使用管線 | 運算子分隔的參數是互斥的。

將範例複製到文字編輯器,並在終端機/Shell 中執行命令之前視需要修改。

參數

別名
必要

用於產生 YAML 範本檔案的 別名。指定的 alias 不會限制您可以使用產生之檔案的部署。

例如

mc batch generate myminio replicate
任務類型
必要

要產生 YAML 文件的工作類型。

支援下列值

全域旗標

此命令支援任何 全域旗標

範例

為複寫任務類型產生 yaml 檔案

以下命令會為複寫類型批次任務產生 YAML 藍圖,並將檔案命名為 replicate 並使用 .yaml 副檔名

mc batch generate alias replicate > replicate.yaml
  • alias 取代為要用於產生 yaml 檔案的 alias

  • replicate 取代為要為其產生 yaml 檔案的工作類型。

    :mc batch 支援 replicatekeyrotate 工作類型。

S3 相容性

mc 命令列工具是為與 AWS S3 API 相容而建置,並使用 MinIO 和 AWS S3 進行測試,以驗證預期的功能和行為。

MinIO 不保證其他 S3 相容服務,因為它們的 S3 API 實作是未知的,因此不受支援。雖然 mc 命令 可能 會按照文件運作,但任何此類使用方式的風險由您自行承擔。

任務類型

mc batch 目前支援下列工作任務類型

replicate

您可以使用以下範例設定作為基礎,來建立自己的自訂複寫批次任務

replicate:
  apiVersion: v1
  # source of the objects to be replicated
  source:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX # 'PREFIX' is optional
    # If your source is the 'local' alias specified to 'mc batch start', then the 'endpoint' and 'credentials' fields are optional and can be omitted
    # Either the 'source' or 'remote' *must* be the "local" deployment
    endpoint: "http[s]://HOSTNAME:PORT" 
    # path: "on|off|auto" # "on" enables path-style bucket lookup. "off" enables virtual host (DNS)-style bucket lookup. Defaults to "auto"
    credentials:
      accessKey: ACCESS-KEY # Required
      secretKey: SECRET-KEY # Required
    # sessionToken: SESSION-TOKEN # Optional only available when rotating credentials are used
    snowball: # automatically activated if the source is local
      disable: false # optionally turn-off snowball archive transfer
      batch: 100 # upto this many objects per archive
      inmemory: true # indicates if the archive must be staged locally or in-memory
      compress: false # S2/Snappy compressed archive
      smallerThan: 5MiB # create archive for all objects smaller than 5MiB
      skipErrs: false # skips any source side read() errors

  # target where the objects must be replicated
  target:
    type: TYPE # valid values are "s3" or "minio"
    bucket: BUCKET
    prefix: PREFIX # 'PREFIX' is optional
    # If your source is the 'local' alias specified to 'mc batch start', then the 'endpoint' and 'credentials' fields are optional and can be omitted

    # Either the 'source' or 'remote' *must* be the "local" deployment
    endpoint: "http[s]://HOSTNAME:PORT"
    # path: "on|off|auto" # "on" enables path-style bucket lookup. "off" enables virtual host (DNS)-style bucket lookup. Defaults to "auto"
    credentials:
      accessKey: ACCESS-KEY
      secretKey: SECRET-KEY
    # sessionToken: SESSION-TOKEN # Optional only available when rotating credentials are used

  # NOTE: All flags are optional
  # - filtering criteria only applies for all source objects match the criteria
  # - configurable notification endpoints
  # - configurable retries for the job (each retry skips successfully previously replaced objects)
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "date" # match objects created after "date"
      createdBefore: "date" # match objects created before "date"

      ## NOTE: tags are not supported when "source" is remote.
      # tags:
      #   - key: "name"
      #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'

      # metadata:
      #   - key: "content-type"
      #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'

    notify:
      endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
      token: "Bearer xxxxx" # optional authentication token for the notification endpoint

    retry:
      attempts: 10 # number of retries for the job before giving up
      delay: "500ms" # least amount of delay between each retry

如需每個金鑰的完整文件,請參閱 複寫批次任務參考

keyrotate

您可以使用以下範例設定作為基礎,來建立自己的自訂金鑰輪換批次任務

keyrotate:
  apiVersion: v1
  bucket: BUCKET
  prefix: PREFIX
  encryption:
    type: sse-s3 # valid values are sse-s3 and sse-kms
    key: <new-kms-key> # valid only for sse-kms
    context: <new-kms-key-context> # valid only for sse-kms

  # optional flags based filtering criteria
  # for all objects
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "date" # match objects created after "date"
      createdBefore: "date" # match objects created before "date"
      tags:
        - key: "name"
          value: "pick*" # match objects with tag 'name', with all values starting with 'pick'
      metadata:
        - key: "content-type"
          value: "image/*" # match objects with 'content-type', with all values starting with 'image/'
      kmskey: "key-id" # match objects with KMS key-id (applicable only for sse-kms)
    notify:
      endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
      token: "Bearer xxxxx" # optional authentication token for the notification endpoint
    retry:
      attempts: 10 # number of retries for the job before giving up
      delay: "500ms" # least amount of delay between each retry

如需每個金鑰的完整文件,請參閱 金鑰輪換批次任務參考

expire

您可以使用以下範例設定作為基礎,來建立自己的自訂過期批次任務

expire:
  apiVersion: v1
  bucket: mybucket # Bucket where this job will expire matching objects from
  prefix: myprefix # (Optional) Prefix under which this job will expire objects matching the rules below.
  rules:
    - type: object  # objects with zero ore more older versions
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 70h # match objects older than this value
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before "date"
      tags:
        - key: name
          value: pick* # match objects with tag 'name', all values starting with 'pick'
      metadata:
        - key: content-type
          value: image/* # match objects with 'content-type', all values starting with 'image/'
      size:
        lessThan: 10MiB # match objects with size less than this value (e.g. 10MiB)
        greaterThan: 1MiB # match objects with size greater than this value (e.g. 1MiB)
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object.

    - type: deleted # objects with delete marker as their latest version
      name: NAME # match object names that satisfy the wildcard expression.
      olderThan: 10h # match objects older than this value (e.g. 7d10h31s)
      createdBefore: "2006-01-02T15:04:05.00Z" # match objects created before "date"
      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object including delete markers.

  notify:
    endpoint: https://notify.endpoint # notification endpoint to receive job completion status
    token: Bearer xxxxx # optional authentication token for the notification endpoint

  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: 500ms # least amount of delay between each retry

如需每個金鑰的完整文件,請參閱 過期批次任務參考