There is some official documentation available at https://docs.divio.com/en/latest/how-to/configure-media-custom-domain/ – follow those steps.
You have to create a separate Divio project for the reverse proxy. You can fork or copy the below sample project and then add it to your Divio project as external repository: https://github.com/what-digital/divio-media-reverse-proxy
You can get the Divio DSN for the S3 storage by logging into your website project via SSH and then execute
echo $DEFAULT_STORAGE_DSN , then take that value to update the nginx configs in /nginx.
Then you can deploy the solution to see if it works. Don’t forget to set the
DEFAULT_STORAGE_DSN env variable in the divio control for your main project.