You're not wrong, that exactly how Skyward Sword works. From when I played, I remember some rare cases where Link would hold the sword awkwardly, especially after some prolonged combats. Holding the Wiimote still while it was pointing the sensor bar for some seconds were enough to automatically re-calibrate and correct this whenever it happened...
Avell A70 MOB: Core i7-11800H, GeForce RTX 3060, 16 GB DDR4-3200, Windows 11 (Insider Preview)
ASRock Z97M OC Formula: Pentium G3258, GeForce GT 440, 16 GB DDR3-1600, Windows 10 (22H2)
ASRock Z97M OC Formula: Pentium G3258, GeForce GT 440, 16 GB DDR3-1600, Windows 10 (22H2)
