User-agent: *
Disallow:/cgi-bin/
Disallow:/css/
Disallow:/docs/
Disallow:/js/
Disallow:/spam/