This is a useful guide to getting basic resources.
Getting Rocks, Stick, Rags
Rocks are usually harvested from rocks with a pickaxe. It was tricky because you needed Rocks to make the tool.
Rocks can be harvested from the bottom of the cobblestone. You can harvest Rocks from the stone fence. You can loot with a high probability from a regular zombie.
Harvest the bushes to earn Stick pile or Thatch.
Thatch can be made into Rags.
You can make a tool by making Rags with ropes.