The oil will undoubtedly be absorbed slightly into all of the accummulated crap on the outside of the engine as well, so short of gunking the engine and steaming it off, it will continue to smoke until it has burnt off.
Just keep an eye on it. Worst case, get a wire brush and do to work on the engine block giving it a clean up.