Jekyll2022-12-09T21:21:18-08:00https://webbmaster.com/feed.xmlWebb MasterCryptic notes to future self that may be useful to othersThomas J. WebbFixing wrong charset in Japanese mp3 files2020-06-06T08:44:22-07:002020-06-06T08:44:22-07:00https://webbmaster.com/2020/06/fix-wrong-charset-in-id3<p>I had some mp3s I downloaded from a publisher’s website, but they were all shift-jis encoded but marked as latin1. A common thing in Japan, unicode is still not at complete saturation. This script I completely whipped up is very single-purpose, one-shot. You’ll likely have to modify it if you’re having a similar problem.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">mutagen.id3</span>
<span class="k">def</span> <span class="nf">findMP3s</span><span class="p">(</span><span class="n">path</span><span class="p">):</span>
<span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">os</span><span class="p">.</span><span class="n">listdir</span><span class="p">(</span><span class="n">path</span><span class="p">):</span>
<span class="n">child</span><span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">child</span><span class="p">)</span>
<span class="k">if</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">child</span><span class="p">):</span>
<span class="k">for</span> <span class="n">mp3</span> <span class="ow">in</span> <span class="n">findMP3s</span><span class="p">(</span><span class="n">child</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">mp3</span>
<span class="k">elif</span> <span class="n">child</span><span class="p">.</span><span class="n">lower</span><span class="p">().</span><span class="n">endswith</span><span class="p">(</span><span class="sa">u</span><span class="s">'.mp3'</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">child</span>
<span class="k">for</span> <span class="n">path</span> <span class="ow">in</span> <span class="n">findMP3s</span><span class="p">(</span><span class="s">'.'</span><span class="p">):</span>
<span class="n">id3</span><span class="o">=</span> <span class="n">mutagen</span><span class="p">.</span><span class="n">id3</span><span class="p">.</span><span class="n">ID3</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">id3</span><span class="p">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="n">value</span><span class="p">.</span><span class="n">encoding</span><span class="o">!=</span><span class="mi">3</span><span class="p">:</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">value</span><span class="p">.</span><span class="n">text</span><span class="p">)):</span>
<span class="n">value</span><span class="p">.</span><span class="n">text</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">=</span> <span class="n">value</span><span class="p">.</span><span class="n">text</span><span class="p">[</span><span class="n">i</span><span class="p">].</span><span class="n">encode</span><span class="p">(</span><span class="s">'latin1'</span><span class="p">).</span><span class="n">decode</span><span class="p">(</span><span class="s">'shift-jis'</span><span class="p">)</span>
<span class="n">value</span><span class="p">.</span><span class="n">encoding</span><span class="o">=</span> <span class="mi">3</span>
<span class="n">id3</span><span class="p">.</span><span class="n">save</span><span class="p">()</span>
</code></pre></div></div>Thomas J. WebbI had some mp3s I downloaded from a publisher’s website, but they were all shift-jis encoded but marked as latin1. A common thing in Japan, unicode is still not at complete saturation. This script I completely whipped up is very single-purpose, one-shot. You’ll likely have to modify it if you’re having a similar problem.Using lix and openfl in vscode2020-05-28T08:02:57-07:002020-05-28T08:02:57-07:00https://webbmaster.com/2020/05/using-lix-and-openfl-in-vscode<p>Put this in the project’s <code class="language-plaintext highlighter-rouge">.vscode/settings.json</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"lime.executable"</span><span class="p">:</span><span class="w"> </span><span class="s2">"lix run lime"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>Thomas J. WebbPut this in the project’s .vscode/settings.json:Fixing Issue With SSL Traffic on AT&T Router2020-05-21T17:44:22-07:002020-05-21T17:44:22-07:00https://webbmaster.com/2020/05/routing-around-att-router<p>My issue is <a href="https://stackoverflow.com/questions/61893382/issue-with-ssl-traffic-originating-from-home-network-destined-to-home-server-usi">detailed here</a> but I thought I’d summarize here basically how to get around it.</p>
<p>Turns out this is an issue with the BGW210 and port 443. If I do the same setup but on 8443 or something else, https works fine. Looking further, it turns out this is is a chronic issue with this and possibly other routers supplied by AT&T (see <a href="https://forums.att.com/conversations/att-fiber-equipment/port-443/5df01162bad5f2f60648d0aa?page=1">here</a>, <a href="https://forums.att.com/conversations/att-internet-equipment/arris-bgw210700-being-blocked-with-disallowed-wanside-management-service-access/5df03076bad5f2f6062cfbdf">here</a> and <a href="https://forums.att.com/conversations/att-internet-equipment/bgw210-port-forwarding-dropping-most-packets-to-specified-port/5defc724bad5f2f606308b9b">here</a>). Trying to use IP forwarding or DMZ settings on the router doesn’t save whatever’s behind it from whatever filtering it’s doing. None of the firewall settings work.</p>
<p>Since I already have static IP, I was able to bypass the filtering and effectively use the BGW as a modem and my own router as the router by turning on <strong>public subnet mode</strong>. To do this:</p>
<ol>
<li>Get all your static IP, default gateway, netmask settings from AT&T if you don’t already have it.</li>
<li>Go to Home Network -> Subnets & DHCP</li>
<li>Turn <strong>public subnet mode</strong> and <strong>allow inbound traffic</strong> on. Set <strong>primary DHCP pool</strong> to public.</li>
<li>Put the info from AT&T into the fields after that as appropriate. I also don’t set the BGW itself to use any of the static IPs since I’m not using it to do NAT for any servers.</li>
<li>Setup your own router of choice with the settings you want, then turn it off, plug its wan port into one of the BGW’s ethernet ports. Turn it on.</li>
<li>Connect server to this router rather than the BGW and setup NAT on it.</li>
</ol>
<p>Now the BGW’s filtering is bypassed and it’s now your own router’s job to handle NAT for your server. This also offers the advantage that you’re not stuck with the BGW’s limited feature set. If you have a good router with DD-WRT on it, you can setup your own DNS, VPN, etc. And yes, this thoroughly solves the port 443 issue.</p>
<p><strong>Beware that anything you connect via ethernet or wifi to the BGW will be exposed directly to the outside world</strong>. I would only plug in routers with firewalls since, again, you’re bypassing the BGW’s firewall. I kept the wifi turned on in case I need to directly connect for diagnostics, but removed automatic connect to it from all computers/changed wifi password.</p>Thomas J. WebbMy issue is detailed here but I thought I’d summarize here basically how to get around it.Setting up Icecast Streaming server on FreeBSD2020-05-16T09:09:59-07:002020-05-16T09:09:59-07:00https://webbmaster.com/2020/05/icecast-on-freebsd<p>I started with <a href="https://www.vultr.com/docs/radio-streaming-on-freebsd-10-with-icecast-and-ices">these instructions</a> for just the icecast part but updated for FreeBSD 12.1. But left out the source client stuff I didn’t need (IceS is too limited features for me) and added the (very impotant imo!) ssl stuff to keep things secure. Install icecast:</p>
<p><code class="language-plaintext highlighter-rouge">pkg install icecast</code></p>
<p>Enable it:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s1">'icecast_enable="YES"'</span> <span class="o">>></span> /etc/rc.conf
</code></pre></div></div>
<p>Start with default config, then edit</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> /usr/local/etc
<span class="nb">cp </span>icecast.xml.sample icecast.xml
</code></pre></div></div>
<p>I started with the <a href="https://certbot.eff.org/lets-encrypt/freebsd-nginx.html">instructions on certbot’s site</a> and <a href="mediarealm.com.au/articles/icecast-https-ssl-setup-lets-encrypt/">here</a>, however I’m personally using this on an instance that will be spun up and stopped so I won’t bother putting the ssl renewal in cron and just renew manually as needed. When you install certbot, there are better instructions for making renewal automatic, but again I’m not going to do that here. Install certbot:</p>
<p><code class="language-plaintext highlighter-rouge">pkg install py36-certbot</code></p>
<p>So for me, I just manually obtain a cert for the domain (http needs to be accessible for this, not just https):</p>
<p><code class="language-plaintext highlighter-rouge">sudo certbot certonly --standalone</code></p>
<p>Then combine the files to create a pem suitable for icecast.</p>
<p><code class="language-plaintext highlighter-rouge">cat /usr/local/etc/letsencrypt/live/DOMAIN/fullchain.pem /usr/local/etc/letsencrypt/live/DOMAIN/privkey.pem > /usr/local/share/icecast/icecast.pem</code></p>
<p>Go to the <code class="language-plaintext highlighter-rouge">authentication</code> section and change passwords to something unique. I definitely recommend not logging in to admin from a browser until you finish the ssl step. Uncomment the <code class="language-plaintext highlighter-rouge">changeowner</code> section under <code class="language-plaintext highlighter-rouge">security</code>. I would also add values to location and admin to prevent warnings about that. Put the right value in hostname. Go to listen-socket and comment/uncomment/change so that only 443 with ssl on is enabled. Save and exit.</p>
<p>Enable logging by making the right dir with the right ownership:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> /var/log/icecast
<span class="nb">chown </span>nobody:nogroup /var/log/icecast
</code></pre></div></div>
<p>Start icecast:</p>
<p><code class="language-plaintext highlighter-rouge">service icecast start</code></p>
<p>If there are warnings you want to tend to, go ahead and re-edit the config file and restart (<code class="language-plaintext highlighter-rouge">service icecast restart</code>).</p>
<p>I used to use nginx with reverse-proxy but this caused some problems like streams crapping out after an hour. icecast might not have as much built-in security measures but it’s special-purpose made for streaming so I think better overall.</p>
<p>I also found, unfortunately, that mixx <a href="https://bugs.launchpad.net/mixxx/+bug/1517087">still doesn’t support ssl</a> as of this writing. But you can always use jack to connect it to something that can. <a href="https://danielnoethen.de/butt/">Butt</a>, on the other hand, does work. Simply fill in the url, port 443 and the source username and password works for me.</p>
<p><strong>However</strong> I’ve found butt to be very crashy and couldn’t even switch audio source without it freezing in linux. But in looking for alternatives, found something even better! Liquidsoap also supports https icecast2 streaming:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>liquidsoap <span class="s1">'output.icecast(%mp3, host="DOMAIN", port=443, password="PASSWORD", mount="radio", protocol="https", mksafe(playlist("playlist.m3u")))'</span>
</code></pre></div></div>Thomas J. WebbI started with these instructions for just the icecast part but updated for FreeBSD 12.1. But left out the source client stuff I didn’t need (IceS is too limited features for me) and added the (very impotant imo!) ssl stuff to keep things secure. Install icecast:Strengths and Weaknesses of Using Haxe for Audio Development2019-05-10T09:33:35-07:002019-05-10T09:33:35-07:00https://webbmaster.com/2019/05/advantages-disadvantages-haxe-audio<p>At the <a href="https://summit.haxe.org/us/2019/">Haxe Summit</a>, I did a talk about doing audio programming in haxe. <a href="/assets/grig_presentation.pdf">Here are the slides</a>. Edit: <a href="https://www.youtube.com/watch?v=IQs2a2KHlpk">here is a link to the video</a> in case you feel like watching me spill water. And below are elaborated notes for the first two slides.</p>
<h2 id="advantagespotential-for-haxe-as-an-audio-programming-language">Advantages/Potential for Haxe as an Audio Programming Language</h2>
<h3 id="multi-target-multi-environment">Multi-target, multi-environment</h3>
<p>You can get back more return on your time investment by using a versatile language. Is the synth you just made relegated to desktop apps but not the browser or other way around? Whichever way the tech goes, you want to be able to keep your options open with regard to dynamic languages vs. deploying compiled code on dynamic’s turf (emscripten, webassembly).</p>
<h3 id="targets-multiple-languages-with-vibrant-audio-communities">Targets multiple languages with vibrant audio communities</h3>
<p>C++ and C obviously are popular languages for audio development and while haxe’s generated C or C++ is usually not well-suited for using directly from the target language, you can make externs for already existing code in these languages and make use of those in haxe for the cpp and hl targets.</p>
<h3 id="macros-allow-for-compile-time-optimizations-and-checks">Macros allow for compile-time optimizations and checks</h3>
<p>Haxe’s macros are excellent, and one use case is preventing certain operations within a given function. So you can ensure that your function doesn’t have any calls to new, for example. Note that this is a high-level optimization and won’t guarantee that mallocs or frees aren’t happening somewhere under the hood (haxe can’t stop the js interpreter from deciding to garbage collect at just the wrong time).</p>
<h3 id="existing-community-of-creatives-game-programmers">Existing community of creatives, game programmers</h3>
<p>Music/audio development and game development both attract creative types and there’s considerable overlap in the people and the use cases. An existing game development community could be a boon to a burgeoning haxe audio community.</p>
<h3 id="type-system-allows-for-generic-algorithms">Type system allows for generic algorithms</h3>
<p>Like C++, haxe allows template parameters so that the same function can work with different types. Useful when you consider how often you want to write audio code that you want to work on different audio formats. Where type parameters are insufficient, macros can come in to generate code that handles specific types in ways even type parameters can’t.</p>
<h3 id="existing-functionality-for-high-level-operations-such-as-playing-audio-files-and-game-audio">Existing functionality for high-level operations such as playing audio files and game audio</h3>
<p>For game applications, we already have a great deal of functionality already there - bindings or externs for openal and ability to load some audio formats for some targets/environments. OpenAL isn’t commonly used in pro audio applications, but it’s actually pretty cool that you have a partial implementation of the spatialization interface even when targetting the browser.</p>
<h2 id="disadvantages">Disadvantages</h2>
<h2 id="no-midi-io-until-now">No MIDI I/O (until now)</h2>
<p>Haxe has a long history of not providing support for talking to midi ports. There is one lib I found that was ported from AS3 and it only talked to a socket, which in turn would be fed from an external non-haxe application. Now thanks to <code class="language-plaintext highlighter-rouge">grig.midi</code>, we do have midi port support in haxe for cpp, js/webmidi, js/nodejs and python targets. Also recent versions of haxe now include externs for webmidi, which <code class="language-plaintext highlighter-rouge">grig.midi</code> makes use of (previously, I had my own externs made for it).</p>
<h3 id="no-pro-audio-appropriate-audio-io-until-now">No pro audio-appropriate Audio I/O (until now)</h3>
<p>The audio i/o functionality in lime, openfl and heaps is designed with games in mind and they are mostly based on OpenAL or a low-level interface with OpenAL abstraction on top (in the case of heaps). Functionality such as timing information in the callback, ability to query sound cards and capabilities, and setting bitrates are generally non-existent or hard to find. One partial exception is we now have externs for webaudio on the js target.</p>
<h3 id="garbage-collected-language-even-when-targeting-c">Garbage collected language, even when targeting C++</h3>
<p>Garbage collection can be tricky to deal with when doing audio programming. If something is allocating without your knowledge in code that’s called over a hundred times a second (e.g., 48k with 256 buffers = 187.5x/second) then you can have performance issues. There are some workarounds, as always you have to be careful when making audio code in gc languages and it’s not haxe’s fault that you can’t make any guarantees about what’s happening across multiple different gc targets. The hl target provdes some flexibility that may be useful (the ability to specify your own gc) and haxe macros can be made to check for any high-level mistakes such as calling new in a callback. The cpp target also allows some tweaks to how gc is done.</p>
<p>Embedded development is one area where haxe is weak due to the gc. In that space, memory management can become very important. I’m not talking about throwing some apps on a raspberri pi, but programming for much lower power devices, something a larger company with larger sales where the economics favors paying more for engineers in order to pay less per unit. Tests should be done to see how far the compiled targets can be pushed. But the paucity of people using haxe this way means, just as with a lot of the audio stuff, that you’re on your own for now.</p>
<h3 id="fragmented-compiled-targets-hxcpp-vs-hashlink">Fragmented compiled targets (hxcpp vs hashlink)</h3>
<p>This isn’t an issue when making code that should work with either, but adds extra work when providing technology that requires native externs, such as the audio and midi i/o I’ve worked on, which thus far is only present for hxcpp.</p>
<h3 id="no-equivalent-to-stdnumeric_limitstype">No equivalent to <code class="language-plaintext highlighter-rouge">std::numeric_limits<Type></code></h3>
<p>Sounds like a minor gripe, but incredibly annoying when trying to make generic algs that can work on multiple integer types of audio or converters (between integer types or between integer and float types). In C++, it’s fairly easy to write templated functions that can work on multiple int types:</p>
<div class="language-c++ highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">template</span> <span class="o"><</span><span class="k">class</span> <span class="nc">T</span><span class="p">></span>
<span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">float</span><span class="o">></span> <span class="n">convertToFloat</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="n">T</span><span class="o">></span> <span class="n">in</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o"><</span><span class="kt">float</span><span class="o">></span> <span class="n">out</span><span class="p">(</span><span class="n">in</span><span class="p">.</span><span class="n">size</span><span class="p">());</span>
<span class="kt">float</span> <span class="n">min</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">numeric_limits</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">min</span><span class="p">();</span>
<span class="kt">float</span> <span class="n">max</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">numeric_limits</span><span class="o"><</span><span class="n">T</span><span class="o">>::</span><span class="n">max</span><span class="p">();</span>
<span class="kt">float</span> <span class="n">range</span> <span class="o">=</span> <span class="n">max</span> <span class="o">-</span> <span class="n">min</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">size_t</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">in</span><span class="p">.</span><span class="n">size</span><span class="p">();</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
<span class="n">out</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="p">((</span><span class="kt">float</span><span class="p">)</span><span class="n">in</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="n">min</span><span class="p">)</span> <span class="o">/</span> <span class="n">range</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">out</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This might not be feasible on all targets if the underlying representation of the numeric type can vary with interpreter, but should at least be provided where it can be.</p>
<h3 id="some-targets-ill-suited-for-use-from-target-language">Some targets ill-suited for use from target language.</h3>
<p>The only target I personally commonly use that does well in this respect is js. I have a feeling C# may also be in this elite category as of recently. However, two languages I use where it’s not the case are python and C++. Python should be easily fixable and there are some okay workarounds, but the basic problem there is due to its history starting out as a hook into <code class="language-plaintext highlighter-rouge">Compiler.setCustomJSGenerator()</code> where it stuffs everything into one file. So haxe namespaces don’t translate to python namespaces. This might be acceptable in internal projects, but I would never ship something that turns haxe dots into python underscores to <code class="language-plaintext highlighter-rouge">pip</code>. I can manually create wrappers so this is frustrating but minor.</p>
<p>Hxcpp and hashlink are tricker to deal with. C++ is a popular and very important language for audio programming so the ability to make c++ code that c++ coders can use would be very handy. Unfortunately the code that is produced is necessarily different to accomodate haxe’s type system and garbage collection and hence somewhat difficult to interoperate with existing c++ code. Also see earlier complaints about garbage collection. There are of course workarounds such as a thin interface, perhaps exported c functions that are called from the other side. We might be stuck with this situation due to the design of haxe although streamlining the one workaround - c interfaces - might help.</p>
<h3 id="few-libs-for-machine-learning-or-dsp">Few libs for machine learning or DSP</h3>
<p>Audio code tends to involve DSP and at present, utility functions such as ffts, resamplers, etc. Haxe doesn’t have an extensive selection of these things, so at present an audio developer would find themself having to reinvent the wheel in ways they wouldn’t in c++, python or, increasingly, js. A new frontier in audio processing (and image processing, which is the same problem but with another dimension added), is to apply traditional DSP techniques to extract features, then feed them into machine learning algorithms. ML can potentially do awesome things like intelligently inferring missing detail, generating novel sounds, speech synthesis, etc.</p>
<p>It would be great to have the basic DSP stuff ported to pure haxe code and have externs for any highly-optimized already existing functions that would be already present in the environment. As well as implementing basic, easy to implement ml algs such as k-means, knn, naive bayes, etc. in pure haxe and provide externs for what exists in C++ and python.</p>
<h3 id="little-support-for-opus-format">Little support for opus format</h3>
<p>MP3 and vorbis have readers. Of course wav does. But somehow the format haxelib lacks anything for opus, which is unfortunate when it’s such a better format, and meant to be the replacement for vorbis.</p>Thomas J. WebbAt the Haxe Summit, I did a talk about doing audio programming in haxe. Here are the slides. Edit: here is a link to the video in case you feel like watching me spill water. And below are elaborated notes for the first two slides.Non-Intrusively Adding Haxe to Your Javascript Setup2019-03-12T19:03:27-07:002019-03-12T19:03:27-07:00https://webbmaster.com/2019/03/non-intrusively-add-haxe<p>Installing haxe is very easy coming from the javascript side. In fact, the way I recommend to install haxe generally is to use <a href="https://github.com/lix-pm/lix.client">lix</a>, which is an npm package (also available on yarn). The only time I’d advise against that is if you don’t want to have npm on your system.</p>
<p>So if you already have a directory with a <code class="language-plaintext highlighter-rouge">package.json</code> (if not, run <code class="language-plaintext highlighter-rouge">npm init</code>), then you can install lix locally:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm <span class="nb">install</span> <span class="nt">--save</span> lix
</code></pre></div></div>
<p>Generally I recommend installing globally (-g), but you don’t have to to be able to use it. So if you just want to try out haxe in one js project, you can install it locally, then use <code class="language-plaintext highlighter-rouge">npx</code> to run lix. To create a <code class="language-plaintext highlighter-rouge">.haxerc</code> with the latest version of haxe:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npx lix scope create
</code></pre></div></div>
<p>You can also install different versions of haxe if you wish and that will modify <code class="language-plaintext highlighter-rouge">.haxerc</code> accordingly. For example, to get the nightly build:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npx lix <span class="nb">install </span>haxe nightly
</code></pre></div></div>
<p>Note that this will change .haxerc to point to the nightly at the time you run it. You must run this again if you wish to update again, as you’d probably expect. The haxe compiler itself is also available through <code class="language-plaintext highlighter-rouge">npx</code> as <code class="language-plaintext highlighter-rouge">lix</code> provides a shim for that. Same thing for <code class="language-plaintext highlighter-rouge">haxelib</code>. The first time you run haxe, it will download the appropriate version for you automatically:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npx haxe
</code></pre></div></div>
<p>Go ahead and install whichever haxe packages you need (maybe externs for react?) Doing so creates entries under <code class="language-plaintext highlighter-rouge">haxe_libraries</code> and also downloads the packages. The contents of <code class="language-plaintext highlighter-rouge">haxe_libraries</code> <em>do</em> belong under source control. They simply contain metadata about version and where the package is located:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npx lix <span class="nb">install </span>haxelib:react
</code></pre></div></div>
<p>So if you obtained something that already has <code class="language-plaintext highlighter-rouge">haxe_libraries</code> but contains libraries you didn’t install yourself already, you’ll need to let <code class="language-plaintext highlighter-rouge">lix</code> know to download them:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npx lix download
</code></pre></div></div>
<p>You can also ensure that this is ran when you run <code class="language-plaintext highlighter-rouge">npm install</code> by adding it to <code class="language-plaintext highlighter-rouge">package.json</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nl">"scripts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"postinstall"</span><span class="p">:</span><span class="w"> </span><span class="s2">"lix download"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>lix’s <a href="https://github.com/lix-pm/lix.client">github page</a> has more information, but with just what you have here, you can easily sneak haxe into any place designed to work with javascript projects. It’s how I got my haxe-based server-side rendering integrated into netlify. Maybe I’ll post about that soon.</p>
<p>Seeing how well lix brings haxe into javascript’s world makes me wish lix could also be built as a python script (it is written in haxe after all, but relies on node externs). If only…</p>Thomas J. WebbInstalling haxe is very easy coming from the javascript side. In fact, the way I recommend to install haxe generally is to use lix, which is an npm package (also available on yarn). The only time I’d advise against that is if you don’t want to have npm on your system.Automated Testing Haxe Libs on Travis and Gitlab CI2018-11-15T06:53:00-08:002018-11-15T06:53:00-08:00https://webbmaster.com/2018/11/automated-testing-haxe-on-travis-and-gitlabci<p>In being able to target so many other programming languages, Haxe presents a unique challenge for testing. You say your Haxe code is “pure Haxe” and hence compatible with all targets, but to truly know that, you need to actually test on all of the targets. That includes languages I’ve never used like lua and languages I hope to never have to use again like php.</p>
<p>Of course, there is no substitute for real, actual testing but automated tests are better than nothing and it did help me, for example, fix a weird php-only bug I had (which may have been a bug in the compiler). My libs all use <a href="https://github.com/lix-pm/lix.client">lix</a> since I like how it does things and hope its the future. And since it’s an npm package I can just piggyback on already existing stuff for nodejs people.</p>
<p>On github, I hook up my repos to <a href="https://travis-ci.org/">Travis CI</a>. I actually moved all of my haxe grig (awesome audio lib I’m working on, more info later) repos to gitlab, but have it automatically push mirror to github since travis only supports that and so people searching there can find it. I use almost the same config for all my haxe projects (or <a href="https://gitlab.com/haxe-grig/grig.midi/blob/master/.travis.yml">see latest</a>):</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">sudo</span><span class="pi">:</span> <span class="s">required</span>
<span class="na">dist</span><span class="pi">:</span> <span class="s">trusty</span>
<span class="na">language</span><span class="pi">:</span> <span class="s">node_js</span>
<span class="na">node_js</span><span class="pi">:</span> <span class="m">6</span>
<span class="na">os</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">linux</span>
<span class="pi">-</span> <span class="s">osx</span>
<span class="pi">-</span> <span class="s">windows</span>
<span class="na">install</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">npm install -g lix</span>
<span class="na">script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">lix download</span>
<span class="pi">-</span> <span class="s">if [[ "$TRAVIS_OS_NAME" != "windows" ]]; then haxelib run travix python ; fi</span>
<span class="pi">-</span> <span class="s">haxelib run travix node</span>
<span class="pi">-</span> <span class="s">if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then haxelib run travix js ; fi</span>
<span class="pi">-</span> <span class="s">if [[ "$TRAVIS_OS_NAME" != "windows" ]]; then haxelib run travix java ; fi</span>
<span class="pi">-</span> <span class="s">haxelib run travix cpp</span>
<span class="pi">-</span> <span class="s">haxelib run travix cs</span>
<span class="pi">-</span> <span class="s">if [[ "$TRAVIS_OS_NAME" != "windows" ]]; then haxelib run travix php ; fi</span>
<span class="pi">-</span> <span class="s">if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then haxelib run travix lua ; fi</span>
</code></pre></div></div>
<p>I take advantage of travis ci’s recently-added Windows support, but exclude the tests that I know from experience don’t work on Windows (yet) for system config related issues. Travix isn’t strictly necessary but it streamlines testing in the various platforms. In addition to using travis, I also use gitlab’s ci. For now I just use it for Linux (and I’m using Bionic Beaver instead of ole’ Trusty Tahr). For that, I use <a href="https://gitlab.com/haxe-grig/grig.midi/blob/master/.gitlab-ci.yml">this config</a>, almost identical in every repo:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">image</span><span class="pi">:</span> <span class="s">osakared/haxe-ci</span>
<span class="na">before_script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">haxelib install hxcpp</span>
<span class="pi">-</span> <span class="s">haxelib install hxjava</span>
<span class="pi">-</span> <span class="s">haxelib install hxcs</span>
<span class="pi">-</span> <span class="s">lix download</span>
<span class="na">test</span><span class="pi">:</span>
<span class="na">script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml --interp</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -python bin/tests.py && python3 bin/tests.py</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -lib hxnodejs -js bin/tests.js && node bin/tests.js</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -java bin/java && java -jar bin/java/RunTests.jar</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -cpp bin/cpp && ./bin/cpp/RunTests</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -cs bin && mono bin/bin/RunTests.exe</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -php bin/php && php bin/php/index.php</span>
<span class="pi">-</span> <span class="s">haxe tests.hxml -lua bin/tests.lua && lua bin/tests.lua</span>
</code></pre></div></div>
<p>gitlab is flexible in that you can specify the docker image and also configure your own runners. So I made my own image for this purpose (<a href="https://github.com/osakared/haxe-docker-ci">github repo</a> and <a href="https://hub.docker.com/r/osakared/haxe-ci/">docker page</a>). As of the writing, the dockerfile is simply:</p>
<div class="language-Dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> ubuntu:18.04</span>
<span class="k">ENV</span><span class="s"> DEBIAN_FRONTEND=noninteractive</span>
<span class="c"># Install Node.js</span>
<span class="k">RUN </span>apt-get update
<span class="k">RUN </span>apt-get <span class="nb">install</span> <span class="nt">--yes</span> curl gnupg2
<span class="k">RUN </span>curl <span class="nt">--silent</span> <span class="nt">--location</span> https://deb.nodesource.com/setup_6.x | bash -
<span class="k">RUN </span>apt-get <span class="nb">install</span> <span class="nt">--yes</span> nodejs npm
<span class="k">RUN </span>apt-get <span class="nb">install</span> <span class="nt">--yes</span> build-essential software-properties-common python-pycurl python-apt
<span class="k">RUN </span>add-apt-repository ppa:openjdk-r/ppa <span class="nt">--yes</span> <span class="o">&&</span> apt-get update <span class="o">&&</span> apt-get <span class="nb">install</span> <span class="nt">-y</span> <span class="nt">--no-install-recommends</span> openjdk-8-jdk
<span class="k">RUN </span><span class="nv">LC_ALL</span><span class="o">=</span>C.UTF-8 add-apt-repository <span class="nt">--yes</span> ppa:ondrej/php <span class="o">&&</span> apt-get update <span class="o">&&</span> apt-get <span class="nb">install</span> <span class="nt">-y</span> <span class="nt">--no-install-recommends</span> php7.1 php7.1-mbstring
<span class="k">RUN </span>apt-get <span class="nb">install</span> <span class="nt">--yes</span> gcc-multilib g++-multilib python3 mono-devel mono-mcs libglib2.0 libfreetype6 cmake luajit luarocks lua-sec lua-bitop lua-socket libpcre3-dev openjdk-8-jdk openjdk-8-jre
<span class="k">RUN </span>npm <span class="nb">install</span> <span class="nt">-g</span> lix
<span class="k">RUN </span>luarocks <span class="nb">install </span>luasec <span class="o">&&</span> luarocks <span class="nb">install </span>lrexlib-pcre <span class="nv">PCRE_LIBDIR</span><span class="o">=</span>/lib/x86_64-linux-gnu <span class="o">&&</span> luarocks <span class="nb">install </span>luv <span class="o">&&</span> luarocks <span class="nb">install </span>environ <span class="o">&&</span> luarocks <span class="nb">install </span>luautf8
<span class="k">ENV</span><span class="s"> JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64</span>
<span class="k">CMD</span><span class="s"> ["/bin/bash"]</span>
</code></pre></div></div>
<p>You can easily modify this or make your own Dockerfile that extends this (<code class="language-plaintext highlighter-rouge">FROM osakared:haxe-ci</code>).</p>Thomas J. WebbIn being able to target so many other programming languages, Haxe presents a unique challenge for testing. You say your Haxe code is “pure Haxe” and hence compatible with all targets, but to truly know that, you need to actually test on all of the targets. That includes languages I’ve never used like lua and languages I hope to never have to use again like php.My Bitwig and Git Setup2018-08-17T22:23:57-07:002018-08-17T22:23:57-07:00https://webbmaster.com/2018/08/my-bitwig-and-git-setup<p>I spend most of my day on the weekdays working on code. I spend anywhere from zero to three hours on music, tending closer to the zero side of that range. So naturally my instincts when working on music are strongly influenced by how I work on code. I like to be organized. I like to keep things in repositories so I’m not hunting through old hard drives years later wondering where that one recording went. I also like to avoid exotic configurations that are hard to recreate, though this can sometimes come at the cost of spontaneity so I’ll make exceptions to this from time to time at the latter stages of working on a song.</p>
<h3 id="instruments">Instruments</h3>
<p>To avoid configuration annoyances if I need to setup Bitwig and my projects on a new computer, I try to avoid as many external dependencies as possible. I believe any good synth can make an infinite range of sounds and using too many different synths is a sign that you’re not using your synths to their maximum potential. And if you use the built-in synths, that’s one less thing you have to install. It’s also hard to find synths that are on all three of Mac, Windows and Linux. I used to standardize on Korg Legacy Collection MS-20 as my one softsynth but getting that to work under Linux using wine was annoying. Now instead my one softsynth I use regularly is <a href="https://obxd.wordpress.com/">obxd</a> which is open source and runs on everything Bitwig does.</p>
<p>I also keep hardware instruments to a minimum. Synths have infinite possibilities inside of them so no need to be a synth dilettante who just searches through presets. I get intimately familiar with synths and start with the init patch and work from there. For now, the only hardware synth I use regularly is my MS-20 mini. I’ll talk more about hardware with future blog posts because there’s something interesting - and analog - that I’m wiring up. But I like to keep hardware synths to a minimum because ultimately I’ll have to bounce those tracks and that’s audio I have to store. When it’s just softsynths, all I’m storing is settings and midi. Nothing that should clog up a git repo.</p>
<h3 id="sometimes-you-do-need-audio">Sometimes You Do Need Audio</h3>
<p>Okay but I can’t avoid audio completely. I spice up my fm and virtual analog softsynths with some real analog. And sometimes there’s vocals. And owing to my industrial influences, samples of things happening, maybe some not quite savory things, make their way into the mix. These, as I recall from the experience of keeping a game’s code and assets in git isn’t in keeping with the decentralized version control philosophy. So I use <code class="language-plaintext highlighter-rouge">git-lfs</code>. My <code class="language-plaintext highlighter-rouge">.gitattributes</code> file looks like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>*.wav filter=lfs diff=lfs merge=lfs -text
*.aif filter=lfs diff=lfs merge=lfs -text
*.multisample filter=lfs diff=lfs merge=lfs -text
*.bwpreset filter=lfs diff=lfs merge=lfs -text
</code></pre></div></div>
<p>This includes the uncompressed audio formats used by the recordings made in bitwig, plus the <code class="language-plaintext highlighter-rouge">.multisample</code> files which have sample data embedded in them. The <code class="language-plaintext highlighter-rouge">.bwpreset</code>s are probably safe to version control normally but as it’s closed-source software I don’t fully know what’s going on under the hood and I prefer to stay on the safe side with these files that are binary anyway. Who knows, maybe some of these presets do include audio data?</p>
<p>I also recently moved from github to gitlab to take advantage of free private repos. Importing won’t get lfs so I manually changed the remote to the new gitlab one and pushed to it. However, it had a complaint I didn’t quite understand about lfs:</p>
<p><code class="language-plaintext highlighter-rouge">remote: GitLab: LFS objects are missing. Ensure LFS is properly set up or try a manual "git lfs push --all".</code></p>
<p>Apparently I needed to have all the lfs objects on my system and <em>then</em> push them to the new origin. So setting old to be the github repo, I simply did this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git lfs fetch old --all
git lfs push origin --all
</code></pre></div></div>
<p>And good to go!</p>
<h3 id="directory-layout">Directory Layout</h3>
<p>I basically start by turning the <code class="language-plaintext highlighter-rouge">Bitwig Studio</code> folder that Bitwig creates (under Documents, My Documents, etc. depending on your os) into a git repo. That means I’m storing all the custom settings I make, my controller scripts and naturally the projects themselves. But I also keep lyrics and such under the same folder. Partly for historical reasons but why not have the ideas and the music itself side-by-side?</p>
<p>I simply have a lyrics directory filled with lyrics saved as markdown. I can easily edit them on gitlab (or github) and it creates good looking html from it as appropriate. I indent or otherwise mark as code any tabs, etc. I had a site a while back that enabled online collaboration on lyrics and some music collaboration features. Maybe the time to bring the idea back as an extension to github, et al is upon us? It would be cool to be able to click on guitar tabs and hear them played…</p>Thomas J. WebbI spend most of my day on the weekdays working on code. I spend anywhere from zero to three hours on music, tending closer to the zero side of that range. So naturally my instincts when working on music are strongly influenced by how I work on code. I like to be organized. I like to keep things in repositories so I’m not hunting through old hard drives years later wondering where that one recording went. I also like to avoid exotic configurations that are hard to recreate, though this can sometimes come at the cost of spontaneity so I’ll make exceptions to this from time to time at the latter stages of working on a song.FM Synthesis Ratios2018-04-22T09:41:00-07:002018-04-22T09:41:00-07:00https://webbmaster.com/2018/04/fm-ratios<p>Just a reference for myself the kinds of sounds I can get from different ratios in fm synthesis (I’m more into analog, but I almost always end up using both analog or virtual analog and fm synthesis in songs)</p>
<table>
<thead>
<tr>
<th>carrier:modulator</th>
<th>vague description</th>
</tr>
</thead>
<tbody>
<tr>
<td>2:1</td>
<td>Darker</td>
</tr>
<tr>
<td>4:1</td>
<td>Rougher</td>
</tr>
<tr>
<td>3:2 (fifth/7 semitones below)</td>
<td>Metallic/bell-like. Better if modulation level not too high</td>
</tr>
<tr>
<td>3:1 (octave & 7 semitones below)</td>
<td>Same as above</td>
</tr>
<tr>
<td>4:3 (forth/5 sems below)</td>
<td>Metallic but less reedy</td>
</tr>
<tr>
<td>8:3 (forth/octave & 5 sems below)</td>
<td>Same as above</td>
</tr>
<tr>
<td>1:1</td>
<td>Subtle/reedy</td>
</tr>
<tr>
<td>1:2, 1:4, 1:8</td>
<td>Bright</td>
</tr>
<tr>
<td>4:5, 2:5, 1:5 (4 sems)</td>
<td>Open, not chime-like</td>
</tr>
<tr>
<td>3:4 (5 sems)</td>
<td>Dull, kinda chime-like</td>
</tr>
<tr>
<td>3:8, 3:16 (1, 2 oct + 5 sems)</td>
<td>Chime-like</td>
</tr>
<tr>
<td>2:3 (7 sems)</td>
<td>Classic/cliché fm sound</td>
</tr>
<tr>
<td>1:3 (oct+7sems)</td>
<td>Clean but piercing</td>
</tr>
<tr>
<td>1:6 (2oct+7sems)</td>
<td>Pinging metalic</td>
</tr>
<tr>
<td>3:5 (9sems)</td>
<td>Classic bright</td>
</tr>
<tr>
<td>3:10, 3:20 (1,2oct+9sems)</td>
<td>Classic 80s chime tones</td>
</tr>
</tbody>
</table>Thomas J. WebbJust a reference for myself the kinds of sounds I can get from different ratios in fm synthesis (I’m more into analog, but I almost always end up using both analog or virtual analog and fm synthesis in songs)Static Websites with Terraform, Netlify and NS12018-03-22T21:36:00-07:002018-03-22T21:36:00-07:00https://webbmaster.com/2018/03/static-websites-with-terraform-and-netlify<p>I have been administrating servers since I was a teenager and I always have an NAS running at my place that I build that does various other things for me that warrant local presence. And I have some proxy servers I use to thwart the region locking by the ip trolls that run the entertainment industry. But I’m also getting to the point in my life where I want to do as little server administration as possible. Sure, administration has gotten quite a bit <em>easier</em> over the years. But if I can avoid paying for ec2 instances, I will. If I can avoid having to worry about installing all the right versions of interpreters and libs on servers, I shall.</p>
<p>So like many people, I’ve moved a lot of things over to being static, using things like gatsby and jekyll to generate the static content locally and leverage the power of cdns and regional caching to speed up websites. One possible configuration for this is s3+cloudfront+route53. That last part, Amazon’s DNS, is necessary unless you want to go back to the bad old days of having to type www before the domain name.</p>
<p>So my requirements for my static websites are as follows:</p>
<ul>
<li>Bare name must work and be the default</li>
<li>https must work and I want to force https</li>
<li>the costs must not balloon up if my traffic is low enough</li>
</ul>
<p>So with these in mind, I went with netlify, at least for the static websites I’m starting out with. Netlify likes to access your github repos and it can build your static websites for you. That is awesome, however I couldn’t find a way in its interface to specify subdirectories so I could put all my static sites in one repo. Maybe I could have deployed different branches to different sites, but that defeats the purpose of keeping it in one repo for me. So I used the manual deployment of building my site, going into its build or _site subdirectory and uploading with netlify:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>netlify deploy
</code></pre></div></div>
<p>I then went into the interface and manually specified to use my domain for it. Terraform <a href="https://github.com/mitchellh/terraform-provider-netlify">seems to have support for netlify</a> but as I don’t see anything for its dns (which is in beta anyway), I went with the company netlify says they use for dns anyway, <a href="https://ns1.com/">ns1</a>. Terraform is awesome and lets me easily setup things like dns entries, web servers, etc. without the tedium of web-based interfaces. Really important as I actually have a lot of static websites to deal with and may have even more coming soon.</p>
<p>Terraform grabs all the terraform files (.tf) in your directory and applies them, so it’s good to simply name the files based on what they do. As I’m only using it for dns for now, I just make a <code class="language-plaintext highlighter-rouge">ns1.tf</code> file in a directory for my terraform configuration, <code class="language-plaintext highlighter-rouge">infra</code> (doesn’t matter what you call it). I could simply put one configuration in there like this:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_zone"</span><span class="w"> </span><span class="s2">"example"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example.com"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"example_web"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example.com"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"ALIAS"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"alias-domain.netlify.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"www_example_web"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"www.example.com"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"CNAME"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"alias-domain.netlify.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"example_mail"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example.com"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"MX"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"1 aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"5 alt1.aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"5 alt2.aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"10 aspmx2.googlemail.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"10 aspmx3.googlemail.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Replacing <code class="language-plaintext highlighter-rouge">alias-domain.netlify.com</code> with the actual netlify subdomain for your site, which you can get by clicking on its dns settings after entering your domain name in its config. Note that I have an ALIAS record here, a non-standard extention to allow a bare domain to effectively point to another external record’s ip. Netlify also insists on having the www subdomain point to it, which you can simply use a CNAME for. Note that I’m also using gmail so I have the standard mx settings for that in here.</p>
<p>This is fine for just one site, but if you have multiple domains with pretty much the same configuration, modules make it so much easier. So I created a <code class="language-plaintext highlighter-rouge">modules</code> directory under my terraform configuration and under that, a <code class="language-plaintext highlighter-rouge">ns1_for_netlify</code> directory. So I put the same config as above, but modified to accept input in a file in that directory called <code class="language-plaintext highlighter-rouge">main.tf</code>:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_zone"</span><span class="w"> </span><span class="s2">"example"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${var.domain}"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"example_web"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"ALIAS"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${var.alias_domain}"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"www_example_web"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"www.${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"CNAME"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${var.alias_domain}"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">resource</span><span class="w"> </span><span class="s2">"ns1_record"</span><span class="w"> </span><span class="s2">"example_mail"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">zone</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"${ns1_zone.example.zone}"</span><span class="w">
</span><span class="err">type</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"MX"</span><span class="w">
</span><span class="err">ttl</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="mi">60</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"1 aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"5 alt1.aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"5 alt2.aspmx.l.google.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"10 aspmx2.googlemail.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">answers</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">answer</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"10 aspmx3.googlemail.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>We’ll also need a way to get those vars in there, so I created another file called <code class="language-plaintext highlighter-rouge">vars.tf</code> in the same directory:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">variable</span><span class="w"> </span><span class="s2">"alias_domain"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">description</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"The address to point the ALIAS bare and CNAME www records to"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">variable</span><span class="w"> </span><span class="s2">"domain"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">description</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"The domain to serve"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Now back in our ns1.tf file, I’m able to easily create multiple websites. Note that you’ll also need to generate an api key for your nsone.com account and put it in here:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">provider</span><span class="w"> </span><span class="s2">"ns1"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">apikey</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"API_KEY"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">module</span><span class="w"> </span><span class="s2">"example"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">source</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"modules/ns1_for_netlify"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"example.com"</span><span class="w">
</span><span class="err">alias_domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"blahblahblah.netlify.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="err">module</span><span class="w"> </span><span class="s2">"fubar"</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="err">source</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"modules/ns1_for_netlify"</span><span class="w">
</span><span class="err">domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"fubar.biz"</span><span class="w">
</span><span class="err">alias_domain</span><span class="w"> </span><span class="err">=</span><span class="w"> </span><span class="s2">"whoopwhoopwhoop.netlify.com."</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>When using modules, you’ll need to call <code class="language-plaintext highlighter-rouge">terraform get</code> before running <code class="language-plaintext highlighter-rouge">terraform apply</code>. Since I had previously used the more tedious approach before switching to using modules, the first time I ran <code class="language-plaintext highlighter-rouge">terraform apply</code> succeeded in deleting the old config but failed in recreating essentially the same config because the old config was still in the process of being deleted. Waiting a few seconds and trying again, it succeeded. You shouldn’t have issues if you use modules from the get-go but just letting you know not to freak out if it does fail at that step. Just wait and try again.</p>
<p>After applying this, I went back into the netlify config for each of the domains, and added https which it can’t do until the dns is right, then after testing I turned on force ssl. I think it’s good to get everything on ssl these days. I tested the websites relentlessly and tested sending e-mails to each of the domains to make sure.</p>Thomas J. WebbI have been administrating servers since I was a teenager and I always have an NAS running at my place that I build that does various other things for me that warrant local presence. And I have some proxy servers I use to thwart the region locking by the ip trolls that run the entertainment industry. But I’m also getting to the point in my life where I want to do as little server administration as possible. Sure, administration has gotten quite a bit easier over the years. But if I can avoid paying for ec2 instances, I will. If I can avoid having to worry about installing all the right versions of interpreters and libs on servers, I shall.