Integrating GAMFT with CEPH storage cluster
View some of the Frequently Asked Questions to our support staff. Included are some tips and tricks making this forum ideal for users getting started with GoAnywhere MFT. Note: Users can reply to existing topics but only our support staff can add new topics to this forum.
1 post
Page 1 of 1
- Posts: 12
- Joined: Thu Feb 16, 2017 11:20 am
Question:
Can GoAnywhere MFT integrate with CEPH cluster storage? We have a local CEPH storage and would like to use it instead of an S3 resource. CEPH is becoming more and more popular as an alternative to expensive Amazon buckets or on-premise storage solutions.
Answer:
I can confirm that GoAnywhere does work correctly/as designed with CEPH storage (Amazon compatible) buckets. Although we do not setup the initial CEPH cluster, we can integrate it with MFT and use an Amazon S3 Bucket resource to connect to the CEPH cluster.
To create an S3 Bucket from CEPH:
# s3cmd -c /root/.s3cfg mb s3://FTP
WARNING: Could not refresh role
Bucket 's3://FTP/' created
Note: You can also use “—configure” option if you don’t already have a configuration file. This file contains the access keys and the information that defines the bucket.
To upload a file to your bucket:
# s3cmd put anaconda-ks.cfg s3://FTP
WARNING: Could not refresh role
upload: 'anaconda-ks.cfg' -> 's3://FTP/anaconda-ks.cfg' [1 of 1]
1245 of 1245 100% in 1s 682.61 B/s done
-OR-
# s3cmd put /tmp/sensu_client_loaded_files s3://FTP
WARNING: Could not refresh role
upload: '/tmp/sensu_client_loaded_files' -> 's3://FTP/sensu_client_loaded_files' [1 of 1]
29 of 29 100% in 0s 2.32 kB/s done
To list the files in your bucket:
# s3cmd -c /root/.s3cfg ls s3://FTP
WARNING: Could not refresh role
2017-11-27 19:59 1245 s3://FTP/anaconda-ks.cfg
2017-11-27 20:00 29 s3://FTP/sensu_client_loaded_files
Your CEPH server should be running a RADOS gateway (A kind of webserver) and all necessary firewall ports should be open:
# lsof -i :7480
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
radosgw 5131 ceph 27u IPv4 43714 0t0 TCP *:7480 (LISTEN)
Once you have a working bucket and gateway, copy your keys from the CEPH configuration file into a GoAnywhere S3 Bucket resource. Note: RADOSGW does not support HTTPS so some people use a TLS proxy. We recommend bypassing it until you have things working.
Once the resource is defined, you may use the resource within any S3 task as a normal S3 resource.
The connection outlined above was made was using the Bucket-Owner which is seen as bad practice for security reasons. We recommend to create sub-users or IAM users and then copy their keys into the resource.
Can GoAnywhere MFT integrate with CEPH cluster storage? We have a local CEPH storage and would like to use it instead of an S3 resource. CEPH is becoming more and more popular as an alternative to expensive Amazon buckets or on-premise storage solutions.
Answer:
I can confirm that GoAnywhere does work correctly/as designed with CEPH storage (Amazon compatible) buckets. Although we do not setup the initial CEPH cluster, we can integrate it with MFT and use an Amazon S3 Bucket resource to connect to the CEPH cluster.
To create an S3 Bucket from CEPH:
# s3cmd -c /root/.s3cfg mb s3://FTP
WARNING: Could not refresh role
Bucket 's3://FTP/' created
Note: You can also use “—configure” option if you don’t already have a configuration file. This file contains the access keys and the information that defines the bucket.
To upload a file to your bucket:
# s3cmd put anaconda-ks.cfg s3://FTP
WARNING: Could not refresh role
upload: 'anaconda-ks.cfg' -> 's3://FTP/anaconda-ks.cfg' [1 of 1]
1245 of 1245 100% in 1s 682.61 B/s done
-OR-
# s3cmd put /tmp/sensu_client_loaded_files s3://FTP
WARNING: Could not refresh role
upload: '/tmp/sensu_client_loaded_files' -> 's3://FTP/sensu_client_loaded_files' [1 of 1]
29 of 29 100% in 0s 2.32 kB/s done
To list the files in your bucket:
# s3cmd -c /root/.s3cfg ls s3://FTP
WARNING: Could not refresh role
2017-11-27 19:59 1245 s3://FTP/anaconda-ks.cfg
2017-11-27 20:00 29 s3://FTP/sensu_client_loaded_files
Your CEPH server should be running a RADOS gateway (A kind of webserver) and all necessary firewall ports should be open:
# lsof -i :7480
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
radosgw 5131 ceph 27u IPv4 43714 0t0 TCP *:7480 (LISTEN)
Once you have a working bucket and gateway, copy your keys from the CEPH configuration file into a GoAnywhere S3 Bucket resource. Note: RADOSGW does not support HTTPS so some people use a TLS proxy. We recommend bypassing it until you have things working.
Once the resource is defined, you may use the resource within any S3 task as a normal S3 resource.
The connection outlined above was made was using the Bucket-Owner which is seen as bad practice for security reasons. We recommend to create sub-users or IAM users and then copy their keys into the resource.
1 post
Page 1 of 1