Flushing caches is necessary because of soft reset
which doesn't clear caches.

Signed-off-by: Michal Simek <monstr@monstr.eu>
Reviewed-by: Marek Vasut <marex@denx.de>