rbd: prevent calling rbdVolume.Destroy() after an error #4564
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It seems possible that the .Destroy() function is called on a nil pointer. This would cause a panic in the node-plugin. Depending on how far GenVolFromVolID() comes, the returned rbdVolume can be connected. If an error occurs, the rbdVolume should not be connected at all, so call the .Destroy() function in those cases too. Fixes: ceph#4562 Signed-off-by: Niels de Vos <ndevos@ibm.com>
Madhu-1
reviewed
Apr 17, 2024
Comment on lines
+1199
to
+1203
| defer func() { | ||
| if err != nil { | ||
| vol.Destroy() | ||
| } | ||
| }() |
Collaborator
There was a problem hiding this comment.
in case of err the volume is already destroyed above, do we need to destroy it again? if yes , can you please add check check for vol !=nil
Member
Author
There was a problem hiding this comment.
generateVolumeFromVolumeID() should now only return a connected volume if no error occurred. But it is possible that this function itself detects an error below. In that case, the .Destroy() function should be called too.
Destroying a volume that is already destroyed is fine, extra checks for that just make the code more complex.
Comment on lines
+1212
to
+1214
| if vol != nil { | ||
| vol.Destroy() | ||
| } |
Collaborator
There was a problem hiding this comment.
when will we hit this one? In case of error we are already doing the Destory and in case of non-nil error case the vol is returned at line 1197
Member
Author
There was a problem hiding this comment.
It is not clear to me if this can be hit, better destroy something that could have been at line 1194.
If the vol was connected, and generateVolumeFromMapping() overwrites that variable, the 1st vol would leak and no .Destroy() would be done on it.
Member
Author
|
/test ci/centos/mini-e2e-helm/k8s-1.29 |
Member
Author
Failed with a panic: |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions. |
Contributor
|
This pull request now has conflicts with the target branch. Could you please resolve conflicts and force push the corrected changes? 🙏 |
1 similar comment
Contributor
|
This pull request now has conflicts with the target branch. Could you please resolve conflicts and force push the corrected changes? 🙏 |
|
This still happens on v3.14.2 by simply following the ceph rbd nomad tutorial: https://docs.ceph.com/en/reef/rbd/rbd-nomad, surprised this has not been taken more seriously. Instead of deferring and causing possible unforeseen issues, could it instead just check on error if rv, err = GenVolFromVolID(ctx, volID, cr, req.GetSecrets())
if err != nil {
if rv != nil {
rv.Destroy()
}
log.ErrorLog(ctx, "error generating volume %s: %v", volID, err)
return nil, status.Errorf(codes.Internal, "error generating volume %s: %v", volID, err)
} |
Collaborator
|
@caquillo07 #5446 should fix the problem and we will get it in in next release. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It seems possible that the .Destroy() function is called on a nil
pointer. This would cause a panic in the node-plugin.
Depending on how far GenVolFromVolID() comes, the returned rbdVolume can
be connected. If an error occurs, the rbdVolume should not be connected
at all, so call the .Destroy() function in those cases too.
Fixes: #4562