IIRC Casual levels don't actually match you based on level. I don't remember the source, and might be wrong because it would make sense, but then again this isn't competitive and you cannot lose levels so they aren't an accurate representation of skill anyway.
>mfw I have no face