I see your point too. Clearance is reducing. It takes about 12" plus the space from the rack itself. On the other hand, the swing arm would give me the freedom to fully operate the tailgate.The swing arm appears to move the rack back pretty far without raising it up. I would worry about grounding while on trails.
How long is the extension you have mounted to gain a little more clearance?