Shed feed line

[MarkS57]

Hi
Here is my scenario. I want to install a 20A 240v service to my shed. Hoping to use a defunct SPA panel left by the PO- a 50A CH50SPA on the side of the house.. Recently, I've had landscaping work done that is between the house & shed and while the landscapers were doing that I had them run a 3/4" SCH40 conduit from the SPA panel to the shed. The 50A service to the SPA panel from the house breaker panel is #6/3 w/ ground. 1st off, I'd like to retain the 6/3 to the SPA panel but replace the 50A breaker in the house panel with a 20A. Outside, I want to remove the SPA panel from the side of the house and replace with an LB body to connect to the conduit. I have 12/3 w/ ground UF to run from the house to the shed where I'll put another another LB to enter the shed. Inside the shed I'll re-install the original SPA panel but with a 20A GFI disconnect in place to the original 50A GFI then run to a small panel for further distribution. I know this will work- question is if I'm running a fowl of any code violations. Thanks, Mark