ARM: uniphier: drop ad-hoc early pin-muxing settings

As the UniPhier serial driver had already switched to Drive Model
and the pinctrl drivers are now enabled, these pin-muxing settings
are properly handled by the pinctrl drivers.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>