Remove the parts depending either on disabled CONFIG_SERIAL_MULTI
or ifdefs around CONFIG_SERIAL_MULTI parts since CONFIG_SERIAL_MULTI
is now enabled by default.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Marek Vasut <marek.vasut@gmail.com>
Cc: Tom Rini <trini@ti.com>
Cc: Anatolij Gustschin <agust@denx.de>
Cc: Stefan Roese <sr@denx.de>