Yes indeed. The hard part is knowing when objects are referenced by something you didn't scan. For example you scan the first half of the heap and find some unreferenced objects; how do you know those objects aren't reachable from the second half of the heap? A lot of GC engineering is an attempt to answer that question as efficiently as possible.
And the big caveat to this simple modification is that, if done naively, previously marked objects may become the sole referencers of non-garbage without being re-enumerated - potentially leading to the garbage-collection of non-garbage, which would be a serious bug.
To fix this, you might have the live code that runs in-between the partial mark+sweeps mark objects to be re-swept as it runs. This adds overhead and complexity, of course. This code is apparently called write barriers by Ruby's GC:
I'm left wondering what the overhead of such greylisting / write barriers are, compared to the reference counting overhead mentioned by shwestrick in the "Reference count, don't garbage collect" thread:
Slight correction, write barriers are usually more expensive(even far more complex sometimes).
HOWEVER reads outnumbers writes by a large margin of 10:1 in common code (iirc this was measured in Java code some 20 years ago so might vary by language), so most GC implementations only use write-barriers since they're an prerequisite for incremental GC marking.
Read-barriers are a prerequisite for incremental GC moving if you need to compact the heap (many earlier GC's accept pauses for compacting work since they often compact smaller parts of the heap in each pause), Java ZGC introduced incremental compacting in recent years and whilst it hurts computation performance(called "mutator" in papers) more than just write barriers they were positively surprised at how low they managed to get the penalty.
Conventual wisdom is the overhead of write barriers is somewhere around 5-10%. There are tricks you can use to reduce it, though; you can coalesce the barriers for objects together into one block of operations if you have a JIT, only do the write barrier if the GC is in a collecting state (and then the overhead is 0% most of the time!), skip the barrier if you know an object is short lived and doesn't escape a scope, etc.
So maybe I'm missing something but couldn't the algorithm as described lead to an infinite loop where some annoying thread insists on making black objects grey faster than the gc can turn them black again, so that the GC ends up not being able to make progress? I think the fix is that when a black object makes a new reference to a white object, instead of making the black object grey, it should make the white object grey.
There's a variety of ways in which the mutator ("annoying thread") may outrun the collector. These can be resolved by throttling at the point of allocation, "stop the world" collections, or giving up and exploding.
Let's say the references are a lot. Like tens of millions order of magnitude.
Would it make sense to GC sweeping it in chunks?