Deploy Alluxio with Minio
In this recipe we will learn how to setup Minio as a persistent storage layer for Alluxio. The Alluxio documentation for using Alluxio with Minio can be found here.
Alluxio provides memory-speed virtual distributed storage for applications. By using Alluxio with Minio, applications can access Minio data at memory speed and with file system APIs commonly used for big data workloads.
This section describes how to set up Alluxio with Minio assumed to be already running. See the Minio quick start guide for how to set up Minio.
Extract the downloaded Alluxio binary, using the appropriate version and distribution. If Minio is the only storage you are using with Alluxio, the distribution is irrelevant.
tar xvfz alluxio-<VERSION>-<DISTRIBUTION>-bin.tar.gz cd alluxio-<VERSION>-<DISTRIBUTION>
Create a configuration file for Alluxio by copying the template.
cp conf/alluxio-site.properties.template conf/alluxio-site.properties
Modify the Alluxio configuration file appropriately for your deployment. This is an example configuration of running Alluxio locally with Minio.
Assume the Minio server is running at
Assume the Minio bucket you wish to mount in Alluxio is
Assume the Minio access key is
alluxio.master.hostname=localhost alluxio.underfs.address=s3a://<MINIO_BUCKET>/ alluxio.underfs.s3.endpoint=http://<MINIO_HOST:PORT>/ alluxio.underfs.s3.disable.dns.buckets=true alluxio.underfs.s3a.inherit_acl=false aws.accessKeyId=<ACCESS_KEY> aws.secretKey=<SECRET_KEY>
Start the Alluxio server locally.
bin/alluxio-start.sh local -f
3. Using Alluxio with Minio
Files which already reside in the Minio bucket will be available thorugh Alluxio. One way to view them is the Alluxio UI. Applications can access and write data in Minio through the Alluxio namespace.
You can run Alluxio's built in I/O tests to see this in action.
After the tests have run, you will see data which has been written in
THROUGH modes in Minio.