The first thing to determine is where on the shape are the three right angles. There are only two possible cases: the two non-right angles are adjacent, or they are not.
Let’s consider the case where they are adjacent, as above. Clearly t>r and s>p. The only way this is possible is if q is the second shortest length. q is the hypotenuse of a triangle whose other sides are (s-p) and (t-r). One of these will be equal to 2 and the other 3, and so q equals sqrt(13). This is not a whole number, and so therefore this case is eliminated.
In this second case, the dashed line is simultaneously the hypotenuse of q and r, and of t and (s-p). Checking through the cases where t is the shortest, second shortest, or third shortest side, there are no solutions. I’ll demonstrate this by assuming that it is the third shortest. As I’ve shown s>p, we’ll consider q=n, p=n+1, t=n+2, s=n+3, r=n+4.
The square of the dashed line = n^2 + (n+4)^2 = (n+2)^2 + 2^2
Expanding each squared bracket gives:
2n^2+8n+16 = n^2+4n+8
Clearly the left hand side is always greater than the right, for any positive whole number n.
Next let’s consider when t is the longest length:
p=n, q=n+1, r=n+2, s=n+3, t=n+4.
The square of the dashed line = (n+1)^2 + (n+2)^2 = 3^2 + (n+4)^2
n^2+2n+1 + n^2+4n+4 = 9 + n^2+8n+16
n^2 – 2n – 20 = 0
Using the quadratic formula, and eliminating the case when n is negative, we get n = 1+sqrt(21). Clearly not a whole number.
Finally let’s look at the last remaining case, where t is the second longest side. r=n, q=n+1, p=n+2, t=n+3, s=n+4.
The square of the dashed line = n^2 + (n+1)^2 = (n+3)^2 +2^2
2n^2+2n+1 = n^2+6n+13
n^2 – 4n – 12 = 0
n = (4 + sqrt(64))/2 = 6
So we now know that the side lengths are 6,7,8,9,10 starting at r and going anti-clockwise. Finally, to find the area, we work out the area of the trapezium to the left of the dashed line and the triangle to the right: 81+21 = 102. And that’s the answer!