Evan Downing

My DJ Notes

2025-09-13T00:00:00-04:00

This post is about my DJing experience for Brazilian Zouk.

Equipment

Laptop
- MacBook Pro M4
Software
- Serato DJ Pro
- Make sure you get the "Pitch 'n Time" extension tool
Controller
- Roland DJ-202 (it came with Serato DJ Pro)
Backpack
- Rockville 25-Key Case Soft Carry Bag Backpack For 25-Key Keyboards Midi Controllers
- https://www.amazon.com/dp/B079ZS9L7L
Headphones
- Audio-Technica ATH-M50x
- https://www.amazon.com/Audio-Technica-ATH-M50x-Professional-Monitor-Headphones
Laptop stand
- Crane Stand Classic Laptop Stand
- I don't think it's available anymore, but you can find something similar.
Audio Cables
- Hosa, Phone Mono 6.3 Mm to RCA Cable (CPR-202): https://www.amazon.com/dp/B000068O17
- yueton 2 Pack 3.5mm Female to Dual RCA Cable 6 Inch: https://www.amazon.com/dp/B0156F38LG

Where do I get songs from?

Overview

A lot of my songs are old (shared between friends from the days of iTunes and iPods). Others have been purchased from or released for free by various artists (pay attention to their social media 😉).

Over the years, I've collected 500-600 zouk (or zoukable) songs 👀 I'm always listening for new songs I hear from Youtube, Zouk events, TV shows (the Bachelor series is great for new songs), or while walking through the airport.

Feel free to email DJs personally -- they're very kind and responsive and generally love to share or sell their hard work to help you get your Zouk song collection started!

DJs

DJ Kakah: https://www.djkakah.com
DJ Alexy: https://soundcloud.com/dj-alexy-874739139
DJ Calado: https://soundcloud.com/djcalado
BCD Zoukreators: https://www.youtube.com/@BCDZoukreators

Music

You can download songs for free from YouTube: https://github.com/yt-dlp/yt-dlp. It's not the best audio quality, but it's good enough in my experience. In general, dancers won't notice, but other DJs (or audiophiles) will 😉

Strategy

DJ by feel (does what I do sound good?)
Simplicity (don't complicate things -- DJ smarter, not harder)
Practice! Practice! Practice! (I use socials to practice -- no need to practice at home all the time)

Transitioning

Key (harmonic)
Energy (match)
BPM (match)
Find your hot cue
Loop in
Adjust volume/bass/treble/etc.
Transition
Loop out

I've never received formal training. Each DJ's style is different. You'll find your own style and songs you like to play. Just make sure you enjoy it!

If you want a professional class, please check out DJ Kakah's courses. She is an expert. I am not. https://www.djkakah.com/challenges

Picking songs

I'm selfish. I pick songs I like, first and foremost. If I don't like what I'm playing, I can't enjoy it.

Adjust songs to be between 70-80 BPM. Use whatever you think sounds best.

Picking the next song

I look at songs that would harmonize with the current song playing. I listen to a new song on my headphones while the other song is playing live.

How to find songs that harmonize

Songs have keys. You can configure your DJ software (like Serato) to label song keys using the Camelot notation. This makes it easier for you to know which songs will harmonize and which won't.

The next song should have either (but NOT both):

+/- one letter
+/- one number

Play around and see what sounds good to you!

How to find songs with the same "energy"

Trial-and-error. Skip through the song you're previewing. See where it's best to start and end the song. You can even color-code songs with similar energy for yourself to note for your future sets.

How to transition so it's not distracting for dancers

Loop an introduction (beat or vocals). Gradually transition so that the dancers "hear" that loop. Move the volumes, reduce the bass in the current song, increase the base in the next song, decrease the treble in the current song, and exit the next song's loop so it continues.

The Camelot Wheel

Workflow

2024-11-11T00:00:00-05:00

Introduction

This post discusses my workflows. I am sharing it in case it is helpful for others to read.

Monday - Thursday

Administrative

Read unread emails and unread Slack messages. Respond to everything.
Check personal goals and their progress for the year.

Gutcheck Projects

Review each project's motivation, goals, "secret sauce", current status, and timeline to finish. Do they make sense?
Heilmeier Catechism
- What are you trying to do? Articulate your objectives using absolutely no jargon.
- How is it done today, and what are the limits of current practice?
- What is new in your approach and why do you think it will be successful?
- Who cares? If you are successful, what difference will it make?
- What are the risks?
- How much will it cost?
- How long will it take?
- What are the midterm and final "exams" to check for success?
Writing papers
- What problem are you solving?
- Why is it an important problem?
- What are the existing approaches? Why are they not sufficient?
- What is your approach? Why is it more effective, efficient, novel, etc?
- How do you implement it? Why is that a good and practical way?
- How do you evaluate your approach and system? Why is the evaluation fair and realistic? How do the results support the goals and claims of your approach and system?

Read academic papers and blog posts

For papers, create a summary:
- Problem
- Existing approaches and their limitations
- Approach
- Evaluation

Coding

Review Pull Requests.
Work on code for each project.
Automate linting and tests via Github actions.

Friday

Start of Day

Administrative
Gutcheck Projects

Write

Brainstorm research ideas
- Focus on defining the problem. If you define the problem well enough, the solution will become clear.
Work on the draft of academic paper or client report
- This is an up-to-date version of the final draft. The weekly meetings serve as snapshots of the final draft, so you can track the evolution of the project.
- The story for the project (motivation)
- Outline the evaluation (how you will measure success)
- Fill in results as you get them each day (if you were successful)
- Working on the final draft of the paper or report helps you not leave things until the last minute
- Save filling in the rest (introduction, related works, discussion) until the week before you submit
- Two days before the deadline, every co-author should have read the final draft and made comments/edits

Starting a New Project

Create a Google Drive of Documents and Slides:
- Related Works (Folder containing PDFs of papers referenced in Background)
- Background (Doc which lists and summarizes related works and existing solutions)
- Brainstorming (Doc containing ideas I have)
- Meetings (Doc of meeting minutes of me presenting the idea to others)
- Overview (Doc of Heilmeier Catechism for planning, Ongoing Summary for up-to-date status of project. See Gutcheck Projects)
- Slides (Presentation during meetings with status updates)
Go Fast!
- To start, search Google Scholar and Scholar Labs for existing works in top-tier conferences. Put these in your Related Works folder -- just download, drag, and drop.
- Use NotebookLM. Add your Background, Brainstorming, and Overview Docs as well as all papers in Related Works/ Ask it research questions, brainstorm ideas, list experiments to perform (and in what order, such that each tells you the most information about what to do next), etc.
Go Slow!
- Read through claims from LLMs about novel research directions and summaries of prior works. Fact check everything. Carefully craft your research direction and list of experiments to perform.
- Create rapid software prototypes and measurements against the state-of-the-art. Ask NotebookLM to craft prompts to give to LLMs to quickly develop these prototypes -- double-check each implementation carefully.

Writing Proposals

Create an overview diagram of what the system will look like. What happens first, second, third, etc. in the pipeline. What is the input? What is the output?
Write & submit an abstract describing the solution. What is the goal? What are the constraints? What is the approach? What are the expected results?
Fill in the front and back matter on the proposal. Now you know how many pages you have to work with to write the technical content.
Create a real-world motivating example (1-2 paragraphs) you will use throughout the proposal.
- This is a framing device to go back and talk about how the solution will handle each challenge in the motivating example.
Split the system into components (which should already be apparent by the diagram) and assign engineers to tackle each part.
Make sure you include the exact metrics the BAA is asking for and say how your solution will accomplish them.

Leadership

Break each project down into 2 or 3 parts, and assign each person one part to own.
For each project, have an end-goal in mind and an end-date. I.e., what specifically do you want to accomplish by when?
Everyone self-updates their progress for you to present at status updates, as well as allows you to ask and answer questions there (an internal Google Doc).
For each status update, create slides and meeting minutes (shared with the client). Have action items they agree to (ranked by importance) for the next meeting.
For each status update presentation, create ~6 slides to share with the client. That way, it is easy for them to track your progress over time.
- Agenda
- Running list of Insights and Contributions
- 2-3 updates (visual/graph/numbers of results)
- Insights for each update -- What did I learn? What is novel/interesting?
- Next steps -- From what I learned last week, what will I do this week?
If you are away on leave, assign someone to lead in your place, and notify the client who the lead contact will be until you get back. No major changes to the project should be made until you return, unless absolutely necessary.
Overspend on the front end of a project, so you can get far enough along that you can divert your attention if needed (e.g., a proposal needs your attention or you have meetings/trainings/other responsibilities to take care of later).
Every time there is a problem or something does not go according to plan, ask the Five Whys (from The Lean Startup book). Ask "why" 5 times, and you will usually identify the root cause of an issue. Fix the fifth (final) "why" first, then fix the other "whys" if they pop up again. Blame the problem, not the person. If someone makes a mistake, it is our fault it was easy for them to make that mistake.

Retrospective

After each project is finished, do a retrospective for lessons learned.
- What went right? Why did it go right?
- What went wrong? Why did it go wrong? What will we do to prevent this in the future?

Documenting Malware for Research

2022-09-08T00:00:00-04:00

Introduction

When I first began my PhD, my advisor suggested I go through various malware source codes (released openly on the Internet) to compile, execute, understand, and document them for future use.

This inevitably paid off, in that it helped me more easily create ground-truth for my datasets and experiments.

Over the years, I've built up a repository of malware source code, along with compilation instructions and documentation on how to run and use the malware -- overall, how the malware works on the inside.

I've been asked by various colleagues to open-source this repo to aid in the education and research of malware analysis and detection.

Why is this difficult?

There are many places to get malware source code from. At the time of this writing, these are two of the most popular sources today:

What's great is that these repos contain malware sources as they originally existed when they were leaked or released.

What's not so great is that these malware source repositories are not clean. They contain lots of garbage files (e.g., temporary files, database files, incomplete source files, etc.) and the documentation on how to compile and use them is either incomplete or non-existent.

In addition, not all malware run on the same operating system. There are malware for Windows, MacOS, Linux, Android, and iOS. Some Windows malware assume Windows XP, while others assume Windows 10. Some malware even assume specific software versions, in order to exploit that particular version. Thus any released code must be able to compile and execute the malware as it was intended -- which is not an easy feat. The documentation must include instructions on how to set up complete environments so that the malware will behave as the author expected it to.

My goal is to create a git repo where I can share these same malware source files, but accompany them with good documentation on how to operate them. The sole purpose is to facilitate an easy way for a researcher to test their solutions (e.g., dynamic analysis, static analysis, malware detection, etc.) on real-world malware samples, straight from the source. This allows the researcher complete control over the sample, so they can experiment with it in a safe environment.

What's the plan?

This blog series will dive deep into the history of malware.

Over time I will release more samples as I clean and document them.

Some posts may be out of order chronologically, and some may be incomplete as I add more malware and content.

Consider this series as a constant work-in-progress.

Where can I follow this?

I will be releasing all source code and documentation here: https://github.com/evandowning/usable-malware

I will be blogging about each sample here on my website: https://www.evandowning.com/tag/usable-malware.html

Final thoughts

This is going to be a long process, and will be accomplished during my free time.

What will take time is documenting how a researcher without access to older versions of VisualStudio (which some of these malware require) will compile and use these samples. I will likely need to update the sample sources to be compatible with the newest versions of Visual Studio. I also plan to write CI/CD pipelines to ensure the malware doesn't breaking in the future.

If you have any complaints of incomplete or incorrect documentation, please open an issue in the git repo above.

If you have any contributions, please create a pull request.

I hope this is useful for you.

Things I Use

2021-01-01T00:00:00-05:00

Personal

Web browser: Brave Browser
Email/Calendar/Drive/VPN/Password manager: Proton
Private messaging: Signal
Multi-factor authentication: YubiKey
Transcriber: MacWhisper
Creating screen demos: OSB Studio
Creating terminal demos: asciinema
Writing papers: Overleaf
Writing assistant: Grammarly
Creating figures: Inkscape
Paper organizer & RSS feed reader: Zotero
Time shifting: Timeshifter
Download video/audio: yt-dlp
Genealogy: gramps
Tor: Tor Browser

Software Development

Time tracking: toggl
Bootstrapping & Dotfiles: yadm
IDE: VS Code
Text & Hex editor: vim
Source code version control: git
Project development, CI/CD: github
Python packaging & version management: uv
Python code template: cookiecutter
Configuring Claude Code: claude-code-config
Compiler Explorer

Analysis Tools

Dynamic analysis:
- Automated Sandbox (malware): CAPEv2
- Honeypot: cowrie
- Debugging on Windows: x64dbg
- Debugging on GNU/Linux: gdb with (gdbinit)
- Network traffic collection & analysis: Wireshark
- Fuzzing:
  - C++: fuzztest
  - Java: jazzer
  - Python: atheris
- Concolic execution: angr
Static analysis:
- Unpacking: unipacker
- Disassembling & Decompiling: Ghidra
- Locating security bugs: CodeQL and Semgrep
Pentesting framework: kali

LLM Tools

Interacting with LLMs locally: AnythingLLM
Implementing LLM-based solutions:

Ongoing Skills Development

Programming: leetcode
Reverse engineering Windows binaries: flare-on
Linux Hacking: pwnable
System Design: course
Operating System Engineering: xv6 and course
Presenting Data and Information by Edward Tufte: course

Benchmarking Neo4j

2019-05-19T00:00:00-04:00

Introduction

Neo4j is a graph database implementation. It's used for creating large graphs to efficiently create and query various relationships. For an explanation on the differences between graph databases and relational database (like MySQL and PostgreSQL), see https://neo4j.com/developer/graph-db-vs-rdbms/.

I required Neo4j for one of my projects and one of the issues we were running into was that it was taking a long time to create our massive graphs (10s of millions of nodes with 100s of millions of edges). We had read that Neo4j was completely capable of handling graphs of this size (and larger) but couldn't figure out why our methods of creating our graphs were so slow.

Unfortunately I couldn't find anyone online who had done benchmarking on creating various sizes of graphs using the different ways of creating a Neo4j graph (which I will explain below). So I thought this was a great opportunity to write some code to test the various ways of creating Neo4j graphs and share my results with anyone interested in answering the same question.

Creating Neo4j Graphs the "Normal" Way

When you first read Neo4j tutorials on creating graphs, they probably mention something about MERGE for creating Nodes and CREATE for creating Edges. MERGE is helpful because it will simultaneously check and create a unique node if it does not exist. If MERGE didn't exist, you would have to create some MATCH condition to say "if this node doesn't exist yet, create it". In the graphs for my project, all of the nodes and edges had to be unique.

Our first implementation just contained those simple queries: Create a node and create an edge. That ended up being very slow (taking 40 seconds to create a graph with 1000 nodes and 1000 edges). So the second thing we tried was creating each node and edge in the same query (as opposed to creating each individual node and edge within individual queries). Another thing we added was creating all nodes and edges first and then creating the remaining unconnected nodes at the end. We also tried adding an INDEX to each node as well. This implementation was faster than the first, but still too slow for our needs. Our test graph (1000 nodes and 1000 edges) took roughly 30 seconds to create.

Then we found a query called a CONSTRAINT which allows the user to create some logical rule for the graph database to follow. For us, it was to only accept unique nodes and edges. But this third implementation also wasn't very fast. Finally we tried getting rid of both INDEX and CONSTRAINT to see if that was bogging anything down, but to no avail. Our fourth implementation had failed us.

We then created a batch implementation where we would batch multiple queries into one (a similar technique used on relational databases). Unfortunately this ended up being slower than our first implementation. Needless to say we were frustrated, but knew there must be a solution that Neo4j provides us.

Pause for Consideration

These timings may sound fast to someone with sufficiently small graphs, but it would take us nearly 10 days to create a single graph with 10s of millions of nodes and 100s of millions of edges (which was way too slow for our needs). Our particular application required streaming graph data to our server where we would need to create these large graphs in realtime (and 10 days was barely realtime). Also we noticed that creating the graphs slowed over time as the graph size grew, so our guestimate of 10 days was actually much smaller compared to what we experienced in reality.

Creating Neo4j Graphs the Efficient Way

After scouring the Internet for a day or so, we happened upon two query methods used to create graphs quickly: UNWIND and LOADCSV.

One can think of UNWIND as the proper way to use batching in Neo4j (like we had tried to emulate above for batching for relational databases). Immediately we saw fantastic results. The time it took to create a graph decreased by an order of magnitude (our 1000 node 1000 edge graph took seconds). Unfortunately during our evaluation of our project, it still was slower than realtime after a day or so of running it with our streaming data. It eventually caught up, but after several days of letting it run after our streaming data stopped after 2 weeks.

Finally we found the best implementation. LOADCSV is by far the fastest way to create large graphs. It had been touted as being able to create a 30 million node graph in minutes https://neo4j.com/blog/import-10m-stack-overflow-questions/. When we implemented using it ourselves it decreased our graph creation time down to 0.17 seconds. Another order of magnitude difference!

But we were also curious in knowing how much different types and sizes of graphs affected the creation time. Below are various timing bar charts of our results:

10 nodes, 10 edges

1000 nodes, 1000 edges

1000 nodes, 10 edges (sparse graph)

10 nodes, 1000 edges (dense graph)

Conclusion

If you're creating a large graph in Neo4j, use LOADCSV.

You can check out all of my source code used to perform my experiments to test for yourself: https://github.com/evandowning/neo4j-benchmark

Converting Protonmail's VCF files to import to phone

2019-05-19T00:00:00-04:00

Introduction

Protonmail captured my interest a few years ago as privacy-focused alternative to Gmail.

But one of the problems I faced was my dependence on Google's Contacts service which kept all of my contacts for me stored on my Android device. I wanted to move away from this as well. But I also have an animosity towards having email on my phone.

So without installing Protonmail on my phone, how could I take my contacts stored on Protonmail and put them on my phone?

Well one obvious way was to export Protonmail's contacts to a VCF file and import that onto my phone. Unfortunately, Protonmail's VCF file was using version 4.0, while my phone (Android version 8) was still stuck on VCF version 2.1. And I wasn't the only one with issues with converting VCF file format versions:

But all of the tools I found went from version 2.1 to something newer. I couldn't find any converters from a newer version down to an older version.

Protonmail (at the time of this post) uses version 4.0. So I wrote a converter from version 4.0 (Protonmail) to version 2.1 (what Android still uses).

You can find my converter here: https://github.com/evandowning/vcf-converter

So everytime my Protonmail contacts get updated, I export them to a VCF file, convert it to version 2.1, and transfer it to my Android device to be imported locally.

Enjoy.

Datasets

2019-01-01T00:00:00-05:00

This is a good website for lots of general cybersecurity datasets: http://www.secrepo.com/

Malware Samples

Large dataset: https://github.com/sophos/SOREL-20M
Malware from 2010-today: https://www.vx-underground.org/

Gathering Data

Static features for PE malware classification: https://github.com/elastic/ember
Dynamic features for clustering PE malware: https://github.com/rieck/malheur
Executing samples to collect dynamic behaviors: https://github.com/kevoreilly/CAPEv2

Markov Text Generator

2016-01-16T00:00:00-05:00

For fun, I've recently created my own Markov text generator.

Explanation

A Markov text generator takes a piece of text as input and outputs new text that is seemingly random. It may even make syntactical and/or semantic sense if properly seeded with input text.

It accomplishes this by splitting the inputted text into tokens based on some separation character. This can result in individual characters or sequences of characters (recognized as words if the separation character is a space).

It then creates a chain of these tokens based on the order of their appearance in the text. The length of this chain is chosen by the generator's user. It then adds the token appearing immediately after this chain of tokens to a list that is associated with that particular chain of tokens. Repeating this process for all inputted text creates the Markov chain.

To generate the random text, the program chooses a chain of tokens as the starting point of the output text.

It then chooses a random token from the list of tokens associated with the chain and add that token to the output text.

After, it extends the chain of tokens to include this next token while removing the first token from the chain. This creates a new chain of tokens to then choose another random token from the list associated with the chain.

After some determined stopping point, the generator ceases to create more text.

Example

So, let's say I have the input text "I like bananas and apples and oranges."

Let's say I want to split this text by spaces, isolating the words and character(s) that come immediately after the word.

I also want the chain's length (chain-size from 'Features' above) to be 1.

So the following chain would be created with its associated next tokens:

chain of tokens: list of tokens coming after the chain in the text

"I": ["like"]
"like": ["bananas"]
"bananas": ["and"]
"and": ["apples", "oranges."]
"apples": ["and"]

As you can see, the chain "and" has two different words it sees after it in the sentence.

So, there's a 50% probability that "apples" or "oranges." will come after the word "and" according to this sentence. However, with the other words there is a 100% probability that the next word will be the only one contained in their list.

You can imagine how more text and chains with lengths greater than 1 will lead to more random and original sentences.

Suppose we don't have any separation. Doing this by character makes it more interesting. It will generate more random words, some misspelled because of the random aspects of chain, and will make less semantic sense than creating chains via words (characters separated by a space).

Here is a link to it: https://github.com/evandowning/markov-text-generator

Evan Downing

My DJ Notes

Equipment

Where do I get songs from?

Overview

DJs

Music

Strategy

Transitioning

Picking songs

Picking the next song

How to find songs that harmonize

How to find songs with the same "energy"

How to transition so it's not distracting for dancers

The Camelot Wheel

Workflow

Introduction

Monday - Thursday

Administrative

Gutcheck Projects

Read academic papers and blog posts

Coding

Friday

Start of Day

Write

Starting a New Project

Writing Proposals

Leadership

Retrospective

Documenting Malware for Research

Introduction

Why is this difficult?

What's the plan?

Where can I follow this?

Final thoughts

Things I Use

Personal

Software Development

Analysis Tools

LLM Tools

Ongoing Skills Development

Benchmarking Neo4j

Introduction

Creating Neo4j Graphs the "Normal" Way

Pause for Consideration

Creating Neo4j Graphs the Efficient Way

10 nodes, 10 edges

1000 nodes, 1000 edges

1000 nodes, 10 edges (sparse graph)

10 nodes, 1000 edges (dense graph)

Conclusion

Converting Protonmail's VCF files to import to phone

Introduction

Datasets

Malware Samples

Gathering Data

Markov Text Generator

Explanation

Example

Suggested Reads

Read these first

Papers

Recommended Conferences

Academic conferences

Industry conferences

Talks

Blog Posts

Machine Learning

Machine Learning for Security

Software Supply Chain

RSS feeds

Funding Opportunities

Books

Computer Science

Software Development

Cybersecurity

Other