當時正在使用jenkins自動構造鏡像推送到nexus上的docker鏡像倉庫,突然間就報錯如下,沒法推送,超過重試次數后也是沒法推送:
ERROR: Build step failed with exception
com.github.dockerjava.api.exception.DockerClientException: Could not push image: received unexpected HTTP status: 500 Server Error
at com.github.dockerjava.core.command.PushImageResultCallback.throwFirstError(PushImageResultCallback.java:40)
at com.github.dockerjava.core.async.ResultCallbackTemplate.awaitCompletion(ResultCallbackTemplate.java:94)
at com.github.dockerjava.core.command.PushImageResultCallback.awaitSuccess(PushImageResultCallback.java:53)
at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$Run.pushImages(DockerBuilderPublisher.java:433)
at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher$Run.run(DockerBuilderPublisher.java:330)
at com.nirima.jenkins.plugins.docker.builder.DockerBuilderPublisher.perform(DockerBuilderPublisher.java:463)
at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:112)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1880)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:428)
Build step 'Build / Publish Docker Image' marked build as failure
Finished: FAILURE
登錄到nexus中進行查看,其中發現有一個錯誤信息:給其中一個倉庫進行 rebulid index 時報錯: A database error occurred
通過查看nexus日志:
2019-07-17 14:16:24,076+0800 ERROR [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Exception `4DF5EBB7` in storage `plocal:/opt/nexus/sonatype-work/nexus3/db/config`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
com.orientechnologies.orient.core.exception.OLowDiskSpaceException: Error occurred while executing a write operation to database 'config' due to limited free space on the disk (3152 MB). The database is now working in read-only mode. Please close the database (or stop OrientDB), make room on your hard drive and then reopen the database. The minimal required space is 4096 MB. Required space is now set to 4096MB (you can change it by setting parameter storage.diskCache.diskFreeSpaceLimit) .
DB name="config"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkLowDiskSpaceRequestsAndReadOnlyConditions(OAbstractPaginatedStorage.java:5073)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1729)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:541)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:99)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2908)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2870)
at org.sonatype.nexus.orient.transaction.OrientTransaction.commit(OrientTransaction.java:70)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:67)
at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:200)
at org.sonatype.nexus.transaction.Operations.call(Operations.java:146)
at org.sonatype.nexus.orient.transaction.OrientOperations.call(OrientOperations.java:56)
at org.sonatype.nexus.quartz.internal.orient.JobStoreImpl.execute(JobStoreImpl.java:202)
at org.sonatype.nexus.quartz.internal.orient.JobStoreImpl.storeJobAndTrigger(JobStoreImpl.java:514)
at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:855)
通過查看日志,提示的是磁盤空間不足造成的,至少需要4096 MB的空間,現在只有3152 MB,找到原因后就好處理了
第一種辦法是增加磁盤剩余空間,或者清除一些垃圾數據,釋放一些磁盤空間
第二種辦法是設置storage.diskCache.diskFreeSpaceLimit變量
這種方式的設置地點為:{nexue安裝目錄}/bin/nexus.vmoptions,
-Dstorage.diskCache.diskFreeSpaceLimit=2048
然后再重新給倉庫rebulid index,推送docker鏡像