Common problem caused by many mains-voltage LED (and CFL) lamps being made down to a price and not having any means to discharge accumulated leakage without lighting up briefly. There could be an insulation problem causing excessive leakage, but it is unlikely, since this nuisance behaviour often occurs with just the normal capacitive leakage in the cables.
All cables have capacitance between cores, so even if the insulation resistance is infinite, there will still be a small current passing through the capacitance whenever an AC voltage exists across the cable. The switch drop has one core PL and one core SL with mains across, and will pass a tiny current into the lamp(s) with the switch off. Because the lamps have an electronic power supply built in, they rectify and smooth the incoming supply. The leakage through the switch drop capacitance slowly charges up the smoothing capacitor until the voltage is high enough for the lamp's voltage converter circuit to start up, the lamp lights, the smoothing capacitor is immediately drained and the lamp goes out.
You mention a measurement of 50V from SL-N with the switch off. In theory you can't measure the voltage on an isolated piece of wire with a multimeter, because there's no complete circuit to allow the measuring input of the meter to function. If you do get a reading, it's not very meaningful, as it depends on the input resistance of the meter as much as it does on the wiring under investigation. I.e. one meter might say it's 50V, another 10V, (i.e. what is popularly called 'ghost voltage'.)
But, because leakage from the line consists of a consists of 230V passing a small current through a high impedance, it's the current that is nearly constant, not the voltage (hence why we talk of leakage current, not leakage voltage) and because most multimeters have a standard input resistance of 10 or 11MΩ we can estimate the current flowing through the meter.
50V/10MΩ = 0.005mA.
This will not be the true value of the leakage current because just as there is leakage from PL to SL, there is leakage from SL to N and to E and all over the shop. It just illustrates roughly the order of magnitude of currents we are dealing with.
We can also estimate, very roughly, how much of the time a lamp will light on that current. Suppose a 7W LED lamp with a running current of 30mA at 230V gradually collects the rectified leakage in its smoothing capacitor. As it builds up voltage across its input the leakage current will reduce, so if it's capable of lighting at 115V and tends to hover around this voltage, we might estimate the average lamp input current at 0.003mA. If the lamp flashes on to full brightness at 115V it will use 60mA from the capacitor so there is enough leakage current to light the lamp for 0.003/60 of the time, or 0.18 seconds per hour. If each flash is 10 ms (determined by the capacitance of its smoother) it will flash 18 times per hour or every 3 minutes. Rough figures indeed, a Fermi approximation, but it shows how the small amount of information given can sometimes still provide a useful insight into what is going on.
How to fix it? Yes, test the insulation resistance as a precaution, just in case there is a wet switch or a rodent-damaged cable, but it's unlikely. Then, provide a path to neutral for that 0.005mA of capacitive leakage. That could in theory be a resistor, but as mentioned, a snubber is a popular solution. It's called a snubber because its normal usage is something completely different, but it's really just a capacitor with a small series resistor. The advantage of using a capacitor to dump the leakage current is that when the switch is on, the current passing through the capacitor is still capacitive, i.e. wattless, so it doesn't dissipate heat or increase the energy consumption. The series resistor just limits the peak current during any voltage transients.
