Invoke-RestMethod cmdlet proxy – more on un-escaping forward slashes in URIs

Recently I posted an article showing how to prevent Invoke-RestMethod from unescaping dots and slashes. Soon after that I got a great advice from Dave Wyatt, moderator for powershell.org forum. What he suggested was to create a proxy function around Invoke-RestMethod cmdlet which will disable UnEscapingDotsAndSlashes when the call is made. After some reading about proxy functions in PowerShell [1] [2] (I couldn’t believe how easy it is to create proxy function in PowerShell) I decided on creating a proxy function which will extend Invoke-RestMethod cmdlet with an extra parameter AllowEscapedDotsAndSlashes. Setting it to true will disable UnEscapingDotsAndSlashes for the time the call to the server is made and then switch back to original setting. This approach restricts intrusion to selected call only, minimising chances of breaking other existing code.

You would use it as follow:

Invoke-RestMethod proxy function

Below is the code of Invoke-RestMethod proxy function:

References


1 Extending and/or Modifing Commands with Proxies


2 Proxy Functions: Spice Up Your PowerShell Core Cmdlets

Why you should not add you PowerShell modules to Windows directory

In his presentations Getting Started with PowerShell 3.0 Jump Start & Advanced Tools & Scripting with PowerShell 3.0 Jump Start , Jason Helmick stresses that you should never put you PowerShell module in Windows directory (usually c:\windows\system32\WindowsPowerShell\…) but he doesn’t give a good explanation why you should not do that. I though that it may be useful for some people to understand why it is not such a good idea to do it.

First of all, You do not “own” that directory, Microsoft does. System updates, patches, hotfixes, they all may go and modify Windows directory, removing your files. Microsoft may decide to move that folder to some other place and it will not be a breaking change.

It’s usually not backed up. The obvious places to be backed up are user folders, especially user Documents. Windows folder is not. That means you can loose your modules when data needs to be restored from back.

It may be wiped out during OS upgrade. There is no guarantee that data in Windows folder will be preserved when upgrading Operating System. You may loose your modules.

It is a bad practice to go and mess up somebody’s else space. The Windows folder belongs to Microsoft, the Pester’s module folder belongs to Pester module and the RabbitMQTools module folder belongs to RabbitMQTools project. Unless you are contributing to any of those project you should not add anything there.

So, follow Jason’s advice and DO put your modules in PowerShell subfolder in user’s documents, not in Windows folder.

How to prevent Invoke-RestMethod from un-escaping forward slashes

While working on RabbitMQTools module I came across a problem with un-escaping forward slashes in Urls by Invoke-RestMethod cmdlet. The default Virtual Host on RabbitMQ is called "/" and when invoking Api methods it must be encoded with %2f:

As it turns out, this will only work if PowerShell is using .Net Framework 4.5 or newer. All the previous versions will un-escape the forward slash changing Url to:

which is not recognised by the server and results with exception. The reason for this behaviour is historical and related to security [1]. Microsoft decided to un-escape forward slash and dot characters to prevent malicious attacks. As this was valid years ago, nowadays with the progress of REST APIs, sometimes it is required to use forward slash in the Url.

If you need to support forward slashes in Url when using .NET Framework 4.0, then there is a way to prevent un-escaping by modifying internal flag on UriParser class. Unfortunately, because the field is internal, it has to be done using reflection:

What the code does is checking whether UnEscapeDotsAndSlashes flag is set on the UriParser class and removes it if necessary . This will stop the un-escaping of Uri allowing Invoke-RestMethod cmdlet to call the API with proper Url.

To check whether UnEscapeDotsAndSlashes flag is on run the following code:

The caveat

Unfortunately, there is caveat with this approach. As it will modify setting for the whole Application Domain it means that once executed in PowerShell session it will apply to any consecutive operations on Uri, even those run outside of your module. If you decide to use this approach, please switch it on only when necessary and revert to previous setting once you are done. It is also a good idea to state clearly the fact that the hack is in use.

To see an example of how it is used in RabbitMQTools module, check AddRabbitMQExchange cmdlet.

Thanks

I’d like to thank stimpy77 who authored the C# solution[2] on StackOverflow.

Disclaimer

The propsed solution is invasive and modifies default behaviour of .NET Framework. You may used it at your own risk. The author doesn’t take any responsibility for any effect which are result of using that approach.

References

1. Erroneous URI parsing for encoded, reserved characters, according to RFC 3986 on Microsoft Connect

2. The discussion on StackOverflow which shows the solution in C#.

Working with REST services in PowerShell

With PowerShell 3.0 comes a wonderful Invoke-RestMethod cmdlet which much simplifies using REST services. The cmdlet handles all communication as well as data serialisation/deserialisation. Below gist shows how easy it is to make a call to get some data from the REST service:

Depending on the result type (atom, JSON, XML), the cmdlet will appropriately deserialise data and return richly object (or collection of objects).

Often access to the service requires authorization. In that case all you need to do is pass and instance of the PSCredential object. Depending on the case you can either use Get-Credential cmdlet which will prompt user for credentials, or create new instance of PSCredential object. Below example, taken from RabbitMQTools module (at the time of writing this module is in very early stage), shows how to list all Virtual Hosts registered with RabbitMQ server, using credentials of guest user:

To create new entity in REST service you use PUT method passing extra data in the body. For example, to create new Virtual Host in RabbitMQ with tracing enabled, you will call method PUT passing hosts name in the Url. Notice that variable $body is converted to JSON object and the content type on REST call is specified as JSON. as well:

Deleting entity means calling REST service and using DELETE method:

Summary

As you can see working with REST Api services in PowerShell is very easy. The Invoke-RestMethod cmdlet fully abstracts all communication and serialisation from the developer. The result comes as rich object and can be piped to other cmdlets like select or where.

I’d like to thank Jason Helmick and Jeffrey Snover for great Getting Started with PowerShell 3.0 Jump Start and Advanced Tools & Scripting with PowerShell 3.0 Jump Start modules on Microsoft Virtual Academy which helped me getting started with PowerShell.

And to find more about creating PowerShell modules come back to this blog, where I plan to write about my experiences at creating RabbitMQTools – the PowerShell module for administering RabbitMQ servers.

Learning PowerShell

For some time now I wanted to learn PowerShell. I have seen some samples on the internet, even used one or two but never had a chance to sit down and learn it. Until now. Recently I have joined Microsoft Virtual Academy and one of the first modules I did was Getting Started with PowerShell 3.0 Jump Start followed by Advanced Tools & Scripting with PowerShell 3.0 Jump Start. Both modules are run by Jason Helmick and Jeffrey Snover.

First module introduces PowerShell and explains some basics, such as getting help, how to use pipeline or remoting. It has enough information to start using PowerShell as a consumer. I must say that as a developer I found it very useful, especially part about remoting. The second module goes deeper into the power of PowerShell and is mostly about extending it with own functions or modules. If you are thinking about creating some custom cmdlets, you have to watch this module. As a matter of fact, the material covered in the jump starts was enough for me to go and start creating my own module RabbitMQTools for administering RabbitMQ servers. During next few months I plan to share my discoveries related to writing PowerShell modules, so keep watching my blog.

For anyone who wants to start using PowerShell I strongly recommend watching those modules.