+
Skip to content

Conversation

driusan
Copy link
Collaborator

@driusan driusan commented Feb 12, 2025

This refactors the way AWS S3 buckets are handled so that the stream wrapper registration is added by a middleware. For scripts, the logic of registering a stream wrapper is moved to generic_includes.

This is a first pass at addressing the comment in htdocs/index.php that most of the work of NDB_Client should be done by middleware.

@driusan driusan added Language: PHP PR or issue that update PHP code Difficulty: Medium PR or issue that require a moderate effort or expertise to implement, review, or test Category: Refactor PR or issue that aims to improve the existing code labels Feb 12, 2025
Copy link
Collaborator

@cmadjar cmadjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@driusan I have the following error when I pull the code on my HBCD VM:

[Thu Feb 13 20:31:26.611174 2025] [php:notice] [pid 1555773] [client 10.20.13.136:56208] [CRITICAL] /var/www/loris/src/AWS/Client.php:21: Uncaught TypeError: LORIS\\AWS\\Client::__construct(): Argument #1 ($loris) must be of type LORIS\\LorisInstance, null given, called in /var/www/loris/src/Middleware/AWS.php on line 38 and defined in /var/www/loris/src/AWS/Client.php:21\nStack trace:\n#0 /var/www/loris/src/Middleware/AWS.php(38): LORIS\\AWS\\Client->__construct()\n#1 /var/www/loris/src/Middleware/ContentLength.php(53): LORIS\\Middleware\\AWS->process()\n#2 /var/www/loris/htdocs/index.php(75): LORIS\\Middleware\\ContentLength->process()\n#3 {main}\n  thrown

I checked and we are not overriding the AWS section.

@driusan driusan added the State: Needs work PR awaiting additional work by the author to proceed label Feb 17, 2025
@driusan driusan force-pushed the AWSMiddleware branch 2 times, most recently from 2598074 to 490675e Compare February 17, 2025 19:30
@driusan
Copy link
Collaborator Author

driusan commented Feb 17, 2025

@cmadjar Sorry! Should be fixed in the latest commit (but I also had to rebase it so it might be hard to test with your hbcd vm after #9556..)

@driusan driusan removed the State: Needs work PR awaiting additional work by the author to proceed label Feb 17, 2025
Dave MacFarlane added 5 commits February 18, 2025 14:03
This refactors the way AWS S3 buckets are handled so that the
stream wrapper registration is added by a middleware. For scripts,
the logic of registering a stream wrapper is moved to generic_includes.

This is a first pass at addressing the comment in htdocs/index.php that
most of the work of NDB_Client should be done by middleware.
@cmadjar
Copy link
Collaborator

cmadjar commented Feb 19, 2025

@driusan now I get the following errors:

[Wed Feb 19 10:14:00.699627 2025] [core:error] [pid 61652] [client ::1:52110] AH00037: Symbolic link not allowed or link target not accessible: /Users/cmadjar/Development/Github/Loris/htdocs/fontawesome, referer: http://localhost:8080/api/v0.0.3/candidates/115788/V3/images/demo_115788_V3_t1_001.mnc

@sruthymathew123
Copy link
Contributor

Hi @driusan, So @jeffersoncasimir mentioned that he will be reviewing this PR as he figured out his minio account issues with IT and has raisinbread data set ready on his end. Thanks.

@jeffersoncasimir
Copy link
Contributor

jeffersoncasimir commented Feb 19, 2025

@driusan

I have attempted to test this with MinIO and here are my issues.

I get the following error message:

PHP Fatal error: Uncaught InvalidArgumentException: Missing required client configuration options:\n\nregion: (string)\n\nA "region" configuration value is required for the "s3" service\n(e.g., "us-west-2")

Does anyone know why the region is set to empty string instead of $config->getSetting('AWS_S3_Region')?

I can bypass this error by setting a region. However, this next issue makes it untestable.

By performing the test below below$s3client->registerStreamWrapper() in NDB_Client:
$file = $s3client->getObject([ 'Bucket' => 'jefferson-bucket', 'Key' => 'test', ]);

I get this error:

PHP Fatal error: Uncaught exception 'Aws\\S3\\Exception\\S3Exception' with message 'Error executing "GetObject" on "https://jefferson-bucket.ace-minio-1.loris.ca:9000/test"; AWS HTTP error: cURL error 6: Could not resolve host: jefferson-bucket.ace-minio-1.loris.ca (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

This is because the library is assuming a "DNS-style bucket+hostname:port", while our MinIO structure is currently "hostname:port/bucket"

s3://jefferson-bucket gets translated to the URL in the error above.

@driusan
Copy link
Collaborator Author

driusan commented Feb 20, 2025

@jeffersoncasimir It is set to the empty string for exactly that reason. You use either the region or the hostname for the client.

@jeffersoncasimir
Copy link
Contributor

jeffersoncasimir commented Feb 20, 2025

@driusan

The constructor is saying that region is required.

Should I request from IT that the expected bucket+hostname:port structure resolves correctly?

@driusan
Copy link
Collaborator Author

driusan commented Feb 20, 2025

Hm.. no, that's weird. It should work. It works with the current NDB_Client, and all I did was move the code around.

Copy link
Contributor

@jeffersoncasimir jeffersoncasimir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have managed to find a configuration that works for my MinIO bucket. See #9579.

I approve this PR given that I have successfully tested and confirmed the following:

  • The LORIS-S3-Enabled header gets added
  • I am able to fetch a file from my bucket using LORIS\AWS\Client from a script using generic_includes.php

@driusan driusan merged commit 9d1ff82 into aces:main Mar 10, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Category: Refactor PR or issue that aims to improve the existing code Difficulty: Medium PR or issue that require a moderate effort or expertise to implement, review, or test Language: PHP PR or issue that update PHP code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载