Testing URIs and URLs with PowerShell

Posted on March 2, 2015 by Jeff Hicks in PowerShell with 0 Comments

Recently I posted some revisions to the tools I use to create a troubleshooting toolkit. My preferred methodology is to maintain download information in a CSV file.

My CSV file looks like this:

Naturally new versions of my selected tools will become available over time, which usually means that the older links will no longer work. In some cases, vendors are smart enough to use one link and simply change what the link points to. But sadly not everyone does that, so I need some way to test if the Uniform Resource Identifier (URI) is still valid. I don’t want to try and download the file. I simply want to test. It turns out we can still use Invoke-WebRequest. The trick is to specify a different method other than the default GET. There is an HTTP method called HEAD, which will answer a web request with basically all of the information except the actual body. The HEAD method is designed for exactly what I need to do.

Using the Invoke-WebRequest cmdlet in PowerShell to test the Uniform Resource Identifier (URI). (Image Credit: Jeff Hicks)

Using the Invoke-WebRequest cmdlet in PowerShell to test the Uniform Resource Identifier (URI). (Image Credit: Jeff Hicks)

The response is very quick because it is not trying to download the body, i.e. file. All I get back are the headers that contain the relevant information I need. I can retrieve details from either the RawContent or BaseResponse properties.

Using the RawContent property to obtain detailed information in PowerShell. (Image Credit: Jeff Hicks)

Using the RawContent property to obtain detailed information in PowerShell. (Image Credit: Jeff Hicks)

This makes it easy to get details about the file I might try to download.

Using the BaseResponse property to obtain more detailed information in PowerShell. (Image Credit: Jeff Hicks)

Using the BaseResponse property to obtain more detailed information in PowerShell. (Image Credit: Jeff Hicks)

Sponsored

Let me point out that even though I am using this cmdlet with the HEAD method to test my CSV file, you can use it for any URI or URL as a very fast-testing mechanism. To make your life easier, I went ahead and created a PowerShell tool in the form of an advanced function.

By default the function will give you a true/false answer if the URI exists. Please be careful as some sites might still return a status code of 200 indicating OK. For example, the following returns true even though I fudged the original URI to make it invalid in terms of downloading:

I added a –Detail parameter to write a custom object to the pipeline to make it easier to identify files from my CSV that can’t be downloaded,.

Using test-url with the -Detail parameter. (Image Credit: Jeff Hicks)

Using test-url with the -Detail parameter. (Image Credit: Jeff Hicks)

I know that the ContentType should be “application” something. Most likely this is pointing to a 404 page. My Test-URI gives me the information I need. I can then use PowerShell to do something with it.

I created a test file with some intentionally broken links so that you could see the results

Sponsored

I would then know to research these links and update my source file. I want to stress that I didn’t include this functionality in Test-URI. If I had, then Test-URI would be limited, and I’d be unable to use it for other URL or URI testing. When creating PowerShell tools and scripts, you don’t want to code yourself into a corner. Write objects to the pipeline and use existing PowerShell commands to work with the results as I did above.

Sponsored

Tagged with , , , , ,