Merging assemblies with ILRepack

Typically, in .NET, to merge assemblies you would use ILMerge. ILRepack is another project you can use and that has had updates recently (and a lot of activity on Github).

I’ve found it useful to merge the lot of Dlls and Exe that can compose a simple app into a single executable file (think containers but without Docker), which facilitates using and distributing it. The new .csproj file format is a great simplification and allows for tasks (as in the previous versions) in a smaller more understandable file.

To use ILRepack in a project, here are the steps you can perform to include the task during your build:

1 – add the ILRepack package to the project you wish to merge files in
2 – validate that the AssemblyName element is defined in the property groups in your .csproj file
3 – in the .csproj, add the following target element

<Target Name="ILRepack" AfterTargets="Build">
<!-- The ILRepack task will grab all build ouput assemblies
and merge them into a single exe file for easy distribution and integration
-->
<ItemGroup>
<ILRepackPackage Include="$(NuGetPackageRoot)\ilrepack\*\tools\ilrepack.exe" /> <NetStandardLocation Include="$(NuGetPackageRoot)NETStandard.Library\2.0.1\build\netstandard2.0\ref" />
</ItemGroup>

<Error Condition="!Exists(@(ILRepackPackage->'%(FullPath)'))" Text="You are trying to use the ILRepack
package, but it is not installed or at the correct location" />
<Exec Command="@(ILRepackPackage->'%(fullpath)') /out:$(OutputPath)merged\$(AssemblyName).exe
/wildcards /lib:@(NetStandardLocation) /target:exe $(OutputPath)$(AssemblyName).exe $(OutputPath)*.dll" /> </Target> 3.1 – the NetStandardLocation element is only required if you are targeting it or including a package that targets it. 3.2 – the /lib:@(NetStandardLocation) argument in the Exec command is only required by 3.1 3.3 – adjust the /out argument in the command to whatever you require. 4 – If you want to include the output in a package, define the package elements in a propertyGroup and include: <GeneratePackageOnBuild>true</GeneratePackageOnBuild> <!--we do not want build outputs integrated into the resulting package, only the IL merged file --> <IsTool>True</IsTool> <IncludeBuildOutput>False</IncludeBuildOutput> </PropertyGroup> <ItemGroup> <!-- nuget package files info: --> <!--we do not want build outputs integrated into the resulting package, only the IL merged file --> <Content Include="$(OutputPath)merged\$(AssemblyName).exe" PackagePath="tools\" /> </ItemGroup> And that’s pretty much it. If you run build, ILRepack will be executed and build the merged file, and MSBuild will also wrap it in a NuGet package if you include the packaging step. DevopsPorto Meetup December 2017 Session I recently presented a talk titled “Ramp Up Your Learning Habits in 2018” and intended to challenge the attendees of the December 2017 DevopsPorto meetup group to choose some topics and focus on learning them in 2018. It was a fun presentation for me, given it is a topic dear to me and was also the first time I presented to the group I co-founded with Eduardo Piairo. The slides are available through GitPitch (which I also learned for this session) at https://gitpitch.com/MiguelAlho/LearningChallenge2018. Press ‘S’ to open the speaker deck and find additional notes. The slides are open source and available at https://github.com/MiguelAlho/LearningChallenge2018 . In the repo, I also have a set of notes on some Lynda.com courses by Britt Andreatta about learning that I found useful and which can further aid any efforts to create new learning habits in 2018. Finally, a quick thanks to all those who attended and followed up during the break and at the end of the day, and also thanks to my good friends at Bitmaker Software for hosting the event at the awesome startup house Founders 2 Founders. DevOps Workshop – Delivering changes to applications and databases I’ll be running a workshop on DevOps with my friend Eduardo Piairo at the Regional Scrum Gathering in Porto, Portugal on Dec. 2nd. For anyone attending there are a few important tools to download and install, preferably before the session. I’ve posted this on a dedicated page at : http://www.miguelalho.pt/devops-workshop/ Hope to see you there! Delivering Changes for Applications and Databases I had a chance to do a presentation with my good friend Eduardo Piairo on “Delivering Changes for Applications and Database” at the monthly PortoData meeting. This was a problem we worked on quite a lot at Celfinet, the client whom I worked for previously, where we met. Delivering changes on apps that we want to be able to work on and iterate on fast require an interesting deal of automation, achievable through migrations, automated tests, CI, and CD. We talked about the shared database pattern and the problems we faced with it, solutions to mitigating that problem with source control and migrations, and how moving to more of a microservice strategy helped in new services. We talked about the common tools we used, like Flyway, to perform the migrations. I demoed some aspects from the dev / app side of things, and showed off a simple migrations and integration test strategy, using DbUp and LocalDb, to aid devs on performing db changes fearlessly throughout the product’s iterations. The code repo is at https://github.com/MiguelAlho/Purchases-DbMigration-sample, and I just updated the readme to reflect how the code evolves, as in the demo. The slides are hosted at slideshare ( http://www.slideshare.net/MiguelAlho/delivering-changes-for-applications-and-databases): Things Worth Watching #6 – A Decade of DDD, CQRS, Event Sourcing (Greg Young) I always enjoy watch Greg Young’s presentations. This one, from DDD-Europe 2016, is very interesting as a retrospect of the changes DDD, CQRS and Event Sourcing have permitted in how modeling problems are attacked and solved with these techniques, and what we can expect in the future. Things Worth Watching #5 – Pacts to the Rescue (Beth Skurrie) Beth Skurrie‘s “Pacts to The Rescue” presentation at Sydney API Days 2015 is worth the watch to get an overview of Consumer-Driven-Contracts and the use of Pact to implement it. One thing I was missing, and that I hadn’t seen yet till watching this, was the Pact-Broker – a centralized service to publish pacts to and consume them in the service providers. A .Net implementation of the broker would also be nice, just to simplify integration, modification and ramp-up. I don’t know of any implementation, but at least there is Pact-net to use pacts in a .Net codebase, provided by SEEK-jobs. Battling TFS Builds and Powershell My last few days have been about getting DNX builds to work in TFS. I’ve got a couple of projects / solutions of different types ongoing, between web apps, web APIs and class libraries. I consider a good build system as a fundamental component in the development process, to aid in continuous integration and enable continuous deployment. TFS vNext builds in TFS 2015 present a new setup, and as with any new tech problems arise, and solutions aren’t always easy to find. Existing tasks don’t always cater to my needs, so I’ve been creating a bunch of simple Powershell scripts to fill in the gaps. And since I’m an absolute Powershell noob, well, lots of problems appeared. So, what follows are some problem-solution scenarios I found along the way. TFS build + Powershell : Persist vars between steps The Context: I’ve opted to give GitVersion a try. One of my solutions is composed of class libraries that are reusable in multiple other solutions (base classes, interfaces, abstractions). My distribution mechanism of choice is an internal nuget server. As such versioning the packages is key. I prefer to follow SemVer since it explicits what changes occur, and also garantees an incremental version number on each change. Previously (pre-dnx), I’ve built a bunch of command line tools for my team to use to control the version numbers. This required devs to run a command pre-commit to increment the number in the AssemblyInfo.cs files (or in our case a single shared file). Increments were driven by the dev, who would choose to increment the major, minor or patch component. .Net assemblies include a fourth component – revision, which we would allow to increment if no changes were made but we simply wanted a new build / package (because something went wrong). NuGet updates in projects / solutions require incremental numbers so this was useful to us. Though not difficult, the manual increment step is still a manual step. GitVersion automates this, guaranteeing at least a patch increment on each new build. Eventually, git commit message can drive minor and major increments (using “feature” and “breaking” in the commit message). Alternatively, the GitVersion.yaml file has a next-version property that can be used for that too. The problem: In my build, I have a step (before dnu restore and build) to get the new version number and add it to an environment variable. The idea would be to persist the variable’s value throughout the various build steps (and not across builds or permanently). Unfortunately, using an$env:varname type of attribution in the Powershell script does not store the variables. Calling Get-Variables in the next steps does not present the variable in the list of available variables. This may be possible in the future, and should be possible in VSO (using specific variables), but not yet in TFS 2015. I open an issue that has a short discussing associated: https://github.com/Microsoft/vso-agent-tasks/issues/375 .

The workaround:
To solve this, I took the “easy” but somewhat inelegant route of serializing the object that GitVersion creates to file, and reading / de-serializing the file contents to object wherever I need it. For this too work, I had to put the GitVersion.yaml next to the .git folder (otherwise GitVersion.exe would not produce the correct output). Since the PS scripts are run from the solution folder, I serialized the object to a gitVersionOutput.json file, which shouldn’t interfere with anything else (and will only exist on the build server).

$GitVersionOutput | Out-File gitVersionOutput.json Once I’ve gotten the new version number for the project / solution, I can run through each project.json file in the solution and substitute the version property’s value. Powershell : ConvertTo-Json might not produce the result you want The Context: As mentioned in the previous step, once I’ve got the new SemVer version number, I can substitute the value in each of the project.json files in the solution. This will allow the build to use that value for each of the generated packages. Package and assembly version numbers will automaticaly be incremented, NuGet packages will be correctly versioned and the Git repo can then be tagged. Binaries, packages and repo will be synced to match specific point-in-tine inputs to outputs. The problem: What would seem to be a simple step /powershell script turn out to be a load of work. The idea was to read each project.json with a combination of the Get-Content and the ConvertFrom-Json cmdlet, change the properties value, and rewrite the file with the ConvertTo-Json and Out-File cmdlets. Unfortunately it’s not straightforward. ConvertTo-Json won’t serialize correctly – deep trees will see some subtrees, like the dependencies subtrees converted to string instead of a tree. In other words, running Get-Content -Raw project.json | ConvertFrom-Json | ConvertTo-Json | Out-File project.json doesn’t produce the correct output. What was read as (shortened for brevity to the relevant parts) { "frameworks": { "dotnet": { "dependencies": { "System.ComponentModel": "4.0.0-beta-23109", "System.Data.Common": "4.0.0-beta-23109", "System.Runtime": "4.0.20-beta-23109", "System.Runtime.InteropServices": "4.0.20-beta-23109", "System.Runtime.Serialization.Primitives": "4.0.10-beta-23109" } } } } Gets written as { "frameworks": { "dotnet": { "dependencies": "@{System.ComponentModel=4.0.0-beta-23109; System.Data.Common=4.0.0-beta-23109; System.Runtime=4.0.20-beta-23109; System.Runtime.InteropServices=4.0.20-beta-23109; System.Runtime.Serialization.Primitives=4.0.10-beta-23109}" } } } Notice that dependencies is no longer an object with properties but a single string. The solution: Two things should be considered. To read the json, Get-Content should have the -Raw switch to avoid using any other piped step (such as Out-String); Get-Content -Raw project.json | ConvertFrom-Json works correctly. The other thing, when serializing, is to include the --depth argument before writing the file : ConvertTo-Json --depth 999 | Out-File project.json , to avoid the deeper parts of the tree to be written as a single string value. Deployment – IIS might be looking for x64 apps because of the app pool The context: This one drove a colleague of mine, who has never used IIS, mad. The context was a simple one. Deploying (x-copy) an app to a IIS based server. Our build doesn’t have this step setup yet (but hopefully it will, soon, with OctopusDeploy), and it’s also something kinda one-shot-ish since we were trying to deploy a branch’s snapshot with some mocks for validation. Esetially we had an X-copy to the server, and reference to an x86 version of a dnx beta. The problem: After small changes to IIS (basic stuff), one last error was keeping the the site / app from displaying. IIS was looking for dnx-clr-win-x64.1.0.0-beta5, and not the x86 variant. We did not have, anywhere, a ref to x64. The solution: This was solved with an AppDomain setting. In the used app Domain, there is a setting that allows 32bit apps to run. Changing the flag value to true corrected the problem. Unfortunately, this one was pretty hard to find. To Get there: 1. In IIS choose the appdomain being 2. Choose used and Advanced settings 3. Change “Activate 32 bit applications” in the General properties group to true Powershell – Hidden vars don’t go out to the command line The context: As previously mentioned, GitVersion is being used to calculate a new SemVer version of the artifacts being generated. The last step, after building, testing and pushing packages is to tag the repo, on the commit that generated the version, with the version number. This is generally done using a git push The problem: The origin argument in the command is a URL to the repo, and authentication info is necessary. In the case of TFS repos, a username/password combination is required. git push at the command line generally requests the user and password values in separate interactions in the shell. To avoid these interactions (since in the build it’s not possible or convenient), a single command should be used with all the required info. in this case, the url for origin can contain the user and pass before the url using the following format: git push https://:@ Passwords are sensitive things and we don’t want them coded in the script. We can use build variables (defined in the variables tab of the build job). TFS supports “hidden” variables that allow you to include sensitive info, and hide them from the UI and logs. These vars are generally available as environment variables throughout the build. While you can access normal variables through${env:variableName} references, hidden vars won’t get substituted correctly in the script, failing the authentication.

The Solution:
Build variables can generally be referenced through the ${env:varname} syntax, but hidden vars can’t. They can, though, be passed as arguments to a script. In this case their value is used internally as expected and still hidden (substituted by ****) in the logs. Script arguments are declared in the start of the script and defined in the arguments textbox in the step definition. in this case, the reference is in the form$(varName).

So for my case, I have 3 vars I define in the build – tagUser, tagPass, tagRepoUrl – where tagPass is hidden. At the very beginning of my powershell script I accept 3 arguments (by ordinal reference) which can be locally accessed as $tagUser,$tagPass, and $tagRepoUrl [string]$tagUser = $args[0] [string]$tagPass = $args[1] [string]$tagRepoUrl = $args[2] In the arguments textbox I reference the build vars in order:$(tagUser) $(tagPass)$(tagRepoUrl)

There’s also a GitHub issue for this one: https://github.com/Microsoft/vso-agent-tasks/issues/388

Extra:
I probably can get the repo url through some other mechanism (maybe gitVersion or .git folder files or something) which I may explore and get rid of the tagRepoUrl variable all together. Seems redundant, just haven’t invested the time for that.

Build vNext projects in TFS (on-premise)

Since I’ve been working with vNExt these last few weeks at Celfinet, and having started a brand new solution for some services, one of my main concerns was to get a build setup ASAP. The lesser the amount of dependencies and projects and what not, the easier and faster it is to setup the build. Faster here is mainly for code sync (from source control, in this case, GIT) and the build feedback. The less amount of work that needs to be done, the faster it will be to se the whole job pass or fail, which in the setup phase can be daunting if you have to wait long.

We generally use one of two build systems – TFS or Jenkins. The new build system (also known as Build.VNext or Build.Preview in the menu) was recently made available to the team in our installation. I’m not a fan of the previous build system. The Xaml workflow concept for defining jobs isn’t a bad idea, but it just isn’t a great experience and is really hard to get into and use. I do like Jenkins flexibility – you build is basically just a sequence of steps of anything – but the UI and navigation could really use some work. When I saw the presentations for the new TFS system, I was hooked (even without trying!). The interface is super clean and has all the flexibility that we got from Jenkins. There is still plenty that can be done to make the new TFS build system spectacular, but it clearly is going in a great direction.

Sidenote: I’m quite curious as to why the ASP.Net team hasn’t adopted the TFS / VSO build system. Preference seems to have gone to Travis/ AppVeyor based builds (as presented on GitHub project pages).

Anyway, not everything was a smooth setup in the build system for my projects. Not that it isn’t easy, but it’s still a preview and documentation is scarce, so I naturally hit some obstacles.

1 – DNVM, DNU updates (prerequisite)

First thing to do in the build, and because we’ll need to build using dnu, is to install/update DNVM and install / upgrade DNU to whatever our solution requires. The process is an automated version of my previous post. The powershell script used is similar to the one at https://msdn.microsoft.com/Library/vs/alm/Build/azure/deploy-aspnet5#Createthedefinition. It :

• updates dnu to whatever is in the solution’s global.json, and sets the appropriate environment var to use it.
• restores packages for the solution’s projects.

The only difference is referenced script only restored dependencies for projects within the \src folder. My modified version restores packages for all the projects (or, better yet, all the project.json) in the solution.

Therefore, the following powershell script was added to the solution folder (and source controlled) and added as a first step in the build process.

2 – MSBuild it

Second step was a regular MSBuild step, using the MSBuild task and pointing to the solution file. It’s more than enough and will correctly capture all the projects and dependency graph within it.

3 – Run xUnit tests

This was a painfull one for me. I spent a lot of time trying to use the VSTest task available, with no luck. It would run, sometimes with success (depending on whatever settings I used – I tried so many it’s now hard to tell). Donovan Brown had already written up a post on this which I tried, but it just wouldn’t work for me – the path to the adapters the he mentions wouldn’t work in my installation. So after a lot of struggle I tried inventing a powershell script to recursively find all the test projects under the solution’s \test folder and run dnx . test on all of them. This off course assumes that all the test projects are under \test and that all of them have a test command in the project.json. It’s a bit forced, but for a first effort should be ok. refactoring later on when more info is available will be required! So after the MSbuild step I added another powershell step to run the xUnit tests:

Like the previous one, the script was placed in source control in the solution directory.

Next steps

Build and test was the most important parte of what I wanted in this first approach to the build. Next step will be generating the appropriate artifactes (nuget packages) and deploying to an environment with OctopusDeploy. Hopefully it’ll be pretty straightfoward 😀