amazon web services - s4cmd sync two buckets access denied -

-

i trying sync 2 s3 buckets:

s4cmd --dry-run sync s3://cgl-rnaseq-recompute-fixed/gtex s3://rnaseq.toil.20k/gtex

but getting following error:

[exception] error occurred (accessdenied) when calling listobjects operation: access denied [thread failure] error occurred (accessdenied) when calling listobjects operation: access denied

the source bucket publicly available. second bucket mine , have access it:

[centos@ip-172-30-3-12 data]$ s4cmd ls s3://rnaseq.toil.20k/                  dir s3://rnaseq.toil.20k/gtex/                  dir s3://rnaseq.toil.20k/pnoc/                  dir s3://rnaseq.toil.20k/target/                  dir s3://rnaseq.toil.20k/tcga/

also cannot ls on source bucket using s4cmd can using s3cmd:

[centos@ip-172-30-3-12 data]$ s4cmd ls s3://cgl-rnaseq-recompute-fixed/gtex [exception] error occurred (accessdenied) when calling listobjects operation: access denied [thread failure] error occurred (accessdenied) when calling listobjects operation: access denied  [centos@ip-172-30-3-12 data]$ s3cmd ls --requester-pays s3://cgl-rnaseq-recompute-fixed/gtex                        dir   s3://cgl-rnaseq-recompute-fixed/gtex/ 2016-06-03 17:02    435553   s3://cgl-rnaseq-recompute-fixed/gtex-manifest

what going wrong? suggestions appreciated.

to achieve s3cmd behavior, use wildcards:

s4cmd sync s3://bucket/path/dira/* s3://bucket/path/dirb/

note s4cmd doesn't support dira without trailing slash indicating dira/* rsync supported.

so in case have use.

s4cmd --dry-run sync s3://cgl-rnaseq-recompute-fixed/gtex/* s3://rnaseq.toil.20k/gtex

check documentation s4cmd helpful.

https://github.com/bloomreach/s4cmd


Comments

Post a Comment

Popular posts from this blog

filehandler - java open files not cleaned, even when the process is killed -

-
there java program read files folder, processing , write output files in folder. ran 4 hours , produced few files failed after throwing following error: "too many files" then modified program close (bufferreader , bufferwriter) , started process again. time process failed after starting throwing following error: java.io.filenotfoundexception: <file> (too many open files) then tried running lsof , searched process having no of open files. realized networkmanager. not sure whether should kill process or not. restarted machine. after when ran java program, still got same error. in worst case might possible code still having leaks in terms of not closing everything, still failing after starting process. have feeling lying in system previous run. if start same process in other machine, runs hours before failing. you cannot retain resource between runs of program, if wanted to. leave system without enough resources in theory, though unlikely, restarting...
Read more

java - Suppress Jboss version details from HTTP error response -

-
my jboss verson jboss eap 6.3 when kind of error response sent jboss, adds server information response. following sample row response: http/1.1 400 bad request server: apache-coyote/1.1 content-type: text/html;charset=utf-8 content-length: 2391 date: tue, 19 jul 2016 09:30:46 gmt connection: close jboss web/7.4.8.final-redhat-4 - jbweb000064: error report jbweb000065: http status 400 - org.codehaus.jackson.map.jsonmappingexception: can not construct instance of mybean.type string value 'xyz': value not 1 of declared enum instance names @ [source: org.jboss.resteasy.core.interception.messagebodyreadercontextimpl$inputstreamwrapper@5e421c2b; line: 14, column: 26] (through reference chain: requestargs["request"]) jbweb000309: type jbweb000067: status report jbweb000068: message org.codehaus.jackson.map.jsonmappingexception: can not construct instance of mybean.type string value 'xyz': value not 1 of declared enum instance names @ [source: org.jboss...
Read more

Sass watch command compiles .scss files before full sftp upload -

-
i use winscp download , edit .scss files , sass on linux (on server) compile them .css . after saving file, use: sass scss/style.scss css/style.css , replaces css file compiled sccs. the problem i want skip part return command line after editing scss file, sake of automation , saving time. but, if use watch command: sass --watch scss/style.scss:css/style.css synchronize 2 files, alerts nonexistent css errors pop up: change detected to: scss/style.scss error scss/style.scss (line 232: invalid css after "...ht: bold; line-": expected "{", "") note that when file uploads quicker usual (sometimes happens) watch command job, no errors. this because, when file upload takes time, sass compiles scss file (on server) before uploaded remote folder. thus, compiles part of file, resulting in css errors. is there way set timer watch command waits few seconds after detects changes , before compiling? any other way overcome is, of course, ...
Read more