Steven Den Beste Strawman

>the brick may appear to have reached 0 velocity, but only because the worker "catches' the brick so that the gravity doesnt pull it back down, and if it was going faster than the gravity ability to recover, then it keeps going at same velocity, wouldn't it?

Yes, I would see it as you. If the worker would be able to throw the
brick out of Earths gravity well, once the well is overcome the brick
keeps the same velocity.

If that is one km an hour it is to slow to reach L5 in a reasonable
time frame. If it would be 30 km a second, (which is incredible fast,
since ion drives output velocities are about the same) 150 million km
would be made in about 58 days (if my calculation is correct) but you
would have to stop this mass with this incredible speed and inertia.

Frank