Yup you've pretty much got the idea.
This picture might help.
Let's first make a few assumptions.
1) Let's assume that this polygon is your circle.
2) Each side of the polygon shows the direction our body is moving in at a particular instant in time.
3) The body is moving at a constant speed around the circle.
4) The body is moving anti-clockwise.
If our body has a constant speed (not velocity), whilst travelling around the circle, this means that the speed is the same for every line on the polygon.
We see that after each instant the path changes direction. But the speed stays the same.
Now remember velocity is both speed and direction. So as we move round our polygon we are changing directions; so technically we are experiencing an acceleration. Specifically, angular acceleration.
We have said that our speed on each line of the polygon is the same. Thus we are not experiencing any acceleration with respect to speed. That is we have no linear acceleration.