When you visit a URL, a web server (like Apache or Nginx) typically looks for a default file to display, such as index.html or php.index . If that file doesn't exist, the server has two choices: Return a error. Generate a Directory Listing .
If you are a researcher or just curious about public web archives, you can use search engines with specific operators. index of parent directory