News

May 28, 2023

Python: 5 Ways to List Files in Directory

File and directory-related operations are basic skills for software engineers. This isn’t just copying one file into another folder on your Windows File Explorer, rather it’s understanding how to

File and directory-related operations are basic skills for software engineers. This isn’t just copying one file into another folder on your Windows File Explorer, rather it’s understanding how to conduct automatic batch operations using software functions.

This is a big topic. Today we will dive into one specific problem: How to list all file names under a specific directory. In Python, a directory contains a group of files and subdirectories.

I’ll introduce five ways to list and access files in a Python directory. Each of these methods are used in different scenarios.

One way to list files in a Python directory is to use the os.listdir() method, which is from Python’s OS module:

The above code will print the names of all files and directories under the current path. If you would like to print the results based on another path, just give the os.listdir() function an argument:

If you only want to print all files, the os.path.isfile() will give you a hand:

For directories, there is also a function named os.path.isdir():

It’s simple and useful, but what if it returns a large list? Or what if you only need a specific type of file? Fortunately, Python provides you with plenty of options for more complex scenarios.

More on PythonMerging Lists in Python

You can also list files in a Python directory using walk(), another method from the OS module.

As its name implies, it can “walk” through a directory tree layer by layer. When you call the os.walk() method, it will return a generator. Every time you call the next() method to generate its next value, it will go one layer deeper. The result will be a tuple that includes three items: (dirpath, dirnames, filenames).

For example, if you want to get the names of all folders in the second layer, your code will be as follows:

Instead of retrieving the names of all files, sometimes you might want to get the names of a specific type of file. Since the glob module is able to add regular expressions in a search, it will be your friend for this type of operation:

The above code will list the file names ending with “log”.

Since Python 3.4, there is a module called pathlib, which is helpful as well. With the help of list comprehension tricks, we can use one line of code to generate all file names of the current path:

Surprisingly, the Path() comes with the glob() function, as well. There’s no need to import the glob module explicitly on the top of your Python file.

More on Python5 Pandas Groupby Tricks to Know in Python

The classic os.listdir() function is intuitive but not efficient for large directories that contain a huge amount of files. Therefore, Python 3.5 introduced a new similar function — os.scandir().

Yes, you probably guessed it. This function will return a generator instead of a list of all names. And you can get names as you need. It’s more efficient in situations where you don’t need to get all of the names at once.