Feature request: support IAM Roles for Service Accounts

[ianonavy]

Is your feature request related to a problem? If so, please describe.

Currently, the S3 storage provider uses static credentials pulled from Kubernetes config. This works great for on-premise Kubernetes clusters, but for cluster admins running in AWS EKS the preferred method of authenticating to AWS APIs is IAM Roles for Service Accounts (IRSA). The kserve controller added support for IRSA in 2021 for non-ModelMesh InferenceServices; it would be great if we could use IRSA with ModelMesh as well.

Describe your proposed solution

What I would propose is instead of failing early when the access_key_id is missing, simply allow the S3 client to infer credentials from the environment using the default credential provider chain if the creds are missing from the config.

Describe alternatives you have considered

For now we will use static credentials since they are the only option. We also considered using a local minio instance as a proxy to S3 since minio does support IRSA, but we decided that was too complex.

Additional context

For context, see this user guide from AWS.

[ckadner]

@ianonavy @mateusz-ciesielski -- if this something you are still interested in, would you want to create a PR for it?

[Radwan-Ibrahim]

@ckadner Mateusz created a PR for adding this feature to ibm-cos-sdk-go, could you please help him by reviewing his PR?
IBM/ibm-cos-sdk-go#13

[mateusz-ciesielski]

@ckadner I have a fork with IRSA support working here: mateusz-ciesielski@57f0cd1

But this commit depends on changes in ibm-cos-sdk-go repository, so we would have to merge IBM/ibm-cos-sdk-go#13 first before adding IRSA support to modelmesh-runtime-adapter.