Puzzle of the Week #307 - Descend the Ladder

This puzzle is based on a similar algorithm to last week’s puzzle, but this time the sequence descends from any composite number, and stops when it reaches a number that is not composite (so, either a prime number or the number 1).

At each stage find the largest prime factor of your number and SUBTRACT this plus one from your number to get the next number, for instance: 24 -> 20 -> 14 -> 6 -> 2

There are many, possibly infinitely many, starting numbers that terminate at either 1 or 2, whereas there are no starting numbers that reach 3 or 7. 5 however is a more interesting case, there is a relatively small set of numbers that lead to 5. Can you find them all?