These input enable settings are handled by the pinctrl drivers.

Because the external bus pins are input-enabled by default, on-board
devices such as LED still work fine even with this delayed input
enabling.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>