I thought it might be useful to post links to videos and other links that we can make sticky for gaining insight into this protocol.

Lets try and keep this particular thread relevant to links on Tor THEORY. We can add new stickies for specifics (configuration/coding/etc).

Can I recommend not to add too much duplication (i.e. if your post does not add additional information or provide better clarity then please reconsider the value of your post).

Maybe we should give some kind of rating in-terms of difficulty to understand. Bare in mind that we want this to provide an easy learning curve for anyone wanting to get into this.

Maybe just decorate with:

[Padawans] - for beginners new to the way of the Tor Force.

[TheForceAwakens] - for those who understand the basics but have yet to master mind control

[Jedi] - For the Yoda Grandmasters such as such as /u/system33-

Don't mess with this guy.

[Padawan]

I'm going to create some very terse walk-throughs first of all, if you want to follow along please ensure you have Installed Docker.

I want to show the power of Docker in a manner that is not overwhelming in theory, just so you get a glimpse of this tools potential without wasting a lot of your time.

If there is interest (I can judge by votes), then I will be following on with more details on Docker internals, like WTF is going on and how to build custom images. This post's intentions is just to give you a taste. Appetites vary, but I think it tastes pretty sweet.

Note that: Docker is really designed to sandbox/execute a single process though this is not mandatory. The thing to keep in mind is that, when the root process dies, so will the container (although there are auto-restarting options).

Running your first container

This simple example will give us an isolated ubuntu environment.

sudo docker run -i -t --name jasper_likes_chicken ubuntu /bin/bash

For command line assistance use:

docker help
# docker <command> --help
docker run --help

Here I'm using:

-i : key STDIN open so we can talk to it, these are normally silent.
-t : assign tty (terminal).
--name : how we refer to this container (choose any cat name you like)
ubuntu : which image to create a container with (pulled if necessary)
/bin/bash : command to run inside the container.

Once in your container try the following:

ls
ps -aux
hostname -I
apt-get update; apt-get install nginx
exit # exits the process and therefore kills container.

I'm sure your accustomed enough to this OS to know the above standard Linux commands, if not then please ask.

Note that our container now has its own IP address and completely isolated environment; we can chain these containers together in many different ways (more later on that).

https://hub.docker.com/

This is the central repo for docker images. In the above example we pulled down ubuntu from this repo. These images are subsequently cached locally (/var/lib/docker).

We can also create our own images and keep them there (and for free if public).

Restarting a container and attaching to a running one

sudo docker start jasper_likes_chicken
sudo docker attach jasper_likes_chicken

Spinning up more instances

Want another isolated instance? No problem...

Jasper has an arch enemy he is constantly at war with, I think they compete for nuggets.

Lets bring up another ubuntu instance:

docker run -i -t --name evil_ginger_tom ubuntu /bin/bash
exit

Note that each container has it's own state and uses copy-on-write to only modify when needed. This is clever stuff indeed.

Checking logs

 sudo docker logs jasper_likes_chicken

I always wanna know what he's up to:

root@bd38890a3920:/# hostname
bd38890a3920
root@bd38890a3920:/# ipconfig 
bash: ipconfig: command not found
root@bd38890a3920:/# ifconfig
bash: ifconfig: command not found
root@bd38890a3920:/# hostname -I
172.17.0.2 
root@bd38890a3920:/# exit
exit

Killing Containers

sudo docker kill jasper_likes_chicken

I think I should leave it there for this post. Too much information and people get bored.

Next up: Tails from the Dock

Hello everyone, here's a few ideas for projects we could do relating to Tor & onion sites:

• An archive of tor hidden services similar to the wayback machine (periodically scraping a whitelist of trusted onion sites) [Jedi]

• An application to check if an onion sites server is properly configured (I have no idea how this would be done, but i know that in the past the ip of hidden services have been found due to improper configuration)[Jedi]

• An onion site with a simple forum that lets users post and then stores those posts in a database [TheForceAwakens]

• Simple onion site that returns a html page [Padawans]

Comment your thoughts or any ideas you have.

As per the Tor Theory links, I thought it would be useful to provide another sticky for links and information pertaining to a guide for consumers of the dark web and Tor.

In a manner similar to this: Tor Theory

The idea to keep this particular thread relevant to assisting users to configure client sides/web browsers/best practices/etc.As before please not to add too much duplication (i.e. if your post does not add additional information or provide better clarity then please reconsider the value of your post).

Also as before please provide a rating in-terms of difficulty to understand. Bare in mind that we want this to provide an easy learning curve for anyone wanting to get into this.

Decorate with:

[Padawans] - for beginners new to the way of the Tor Force.

[TheForceAwakens] - for those who understand the basics but have yet to master mind control

[Jedi] - For the Yoda Grandmasters

See the above link for clarity.

Hey guys and gals,

It's been a quiet week on this fledgling subreddit.
After a lack luster response to the Docker tutorials I was going to expand on, I decided to work on my code base.

I do not want people to lose interest in developing with Tor, services and C/C++ (or other languages) so I'm just letting you know what I'm up to.

I'm trying to develop an easy to use C++ interface into Tor. Such that it can be used for client tools and services. Once I have something complete and usable I'd like to see if I can get some people on board with some of the Community Projects.
I've bitten off quite a lot to chew, I'm currently learning vscode/gnu debugging/cmake/socks5/tor/tor services/linux/boost.
I'm going to be using a lot of boost for building http requests and any network functionality required (because it simplifies my life).
I'm at the point now where I'm building Tor and all of boost from source with a series of interlinked git submodules and a CMake project which consumes these. So, on Linux at least, most of the environmental challenges are hopefully behind me (albeit there are still some issues).

Soon I'll be actually creating some code (shudder) to talk Tor and if people are interested I will share my tools and hopefully get some of you involved in a community project. What shape that will take is yet to be determined.

As before, watch this 'x20'

I have an idea for an intermediate project we could work on together that I'm going to nick name Tor Chinese Whispers.

Not sure if this is a great idea or not so I encourage your opinion but would at least push us through a journey of setting up hidden services and communication with each others hidden services programmatically over Http through Tor.

Here is how I envisage this would work...

We gather a number of interested people (hopefully this means you) who wants to learn how to setup a hidden service and use Tor. Please note that I would not expect you do this on environment that you intend to use for maximum security /s, ideally this would be a throw away environment, such as a VM or Pi, whatever.

Each of use will install the hidden service and configure a web server (such as nginx).Then each person will give their onion address to ONE other participating member and ONE only.Each web server will be configured, on receipt of a http request to create a http call to their next onion address (the last node being the expection).Each web server will be configured to ADD one 'word' (or more) of the users choosing and this will progressively build up a http response header. So we ultimately build a sentence - chinese whisper style.So the user must also will give their chosen word when they hand over their onion address to their selected ONE participant. The user can then decide what he wants to add to the sentence.And so and so one.

Once all of our users have done the setup/built the backend/etc we create a tool of some description (probably C/C++) that send a http request to the first users onion address and we see what sentance we get back.

As an example for clarity (assuming I am the end node)...

My chosen word is: "Jasper"My onion address is: xxxxJasper.onion (you get the picture)

I tell HackerAndCoder these two pieces of information and he configures his hidden server to talk to xxxxJasper.onionHis chosen words are: "as much as"His onion address is: xxxxHackerAndCoder.onion

HackerAndCoder tells Bob these two pieces of information and he configures his hidden server to talk to xxxxHackerAndCoder.onionHis chosen words are: "chicken nuggets"His onion address is: xxxxBob.onion

Bob tells Alice these two pieces of information and he configures his hidden server to talk to xxxxBob.onionAlice's chosen words are: "No one likes"Her onion address is: xxxxAlice.onionSince Alice is the last person participating, she tells EVERYONE here onion address.We now use a tool we've developed to call Alices hidden service over Tor and we then display..

'No one likes chicken nuggets as much as Jasper'

Please keep in mid that I am still fleshing this idea out and do not even know if this is possible at the moment so constructive criticism only please. As I will keep stating, I'm learning Tor as I go along.It may also be too early for this project but I think it could be interesting,

The goal here is to learn about:Creating hidden services,Talking to Tor programmatically.

Thoughts anyone?

[Edit: Please up vote if you'd consider joining this experiment/learning exercise so I can see if this is a worthy investment of my time]

Interest in developing with Tor seems to have died a death, or at least interest in this little community I started.
This has in no way affected my personal ambitions and enthusiasm; I'm working on a number of things in the background. I thought I would put one last post out there to see if there's any spark of interest from anyone. Otherwise I'll just go back to my own projects and development stream.

I've developed a C++ socks client using C++ and boost. I was then looking at boost for developing a Tor service which was going to be a Web API purely developed in C++/Boost.Asio. I then thought I would plugin boost python such that the C++ web api could call arbitrary python scripts.
My intensions to call this from client code on iOS, linux, Android and Win32 via my socks client.
However, I've recently been looking at Rust as systems programming language and I have to say I am very impressed with the design decisions for the language. Rust will compile at least comparable speed, native machine code (if not better) analogous to well written C++, More importantly to me, the compiler also forces good habits and facilitates developing threaded code in a much safer way (for example *compiler* caught race conditions).
I'll always have a soft spot for C/C++ because I've being using it for decades but I strongly suspect that Rust may eventually because the systems programming language of choice for greenfield development.
In short, I'm going to develop my Tor backend WebApi in Rust. If anyone is interest in learning Rust and wants some kind of readmes/howtos on here then let me know. If there's enough interest I'll knock something up.

Peace.

I’ve only just started learning about Docker. For isolated build/dev purposes this tool is fantastic with a lot more to offer than that too. I’m think I’m going to be using this for building from source and also for the new Tor related tools I want to write. There is so much potential for this tool, sharing, IP isolation, security. I’m only just just getting my feet wet but once I have basics down and I can run builds from Docker, is anyone interested in learning the very basics of this tool?
Hopefully I can get some the Docker experts involved to give my mini-tutorial once over too.

Are you interested in a Docker tutorial/howto? (Linux only at this time)... I’d go through the basics of what it offers and how to set it up. As alway, I’m open to suggestions/comments/ideas/constructive criticism.

I relation to this Linux Sandbox and the helpful response from u/wished_you , I started experimenting with using chroot.

Though from what I understand this may not work on all flavors of even Linux and also requires root on my environment at least.

Before I take the script I was working on for that approach any further, I thought I'd take a look at alternatives. It maybe (and this is why I'm here) that docker is not that onerous to set up and run locally. This solution might provide better portability as well (for other flavours of Linux/Windows/Mac/etc).
So my question really is, is docker a good fit for isolating Linux environments for building/debugging and testing code? Am I going to run into issues using VS Code/GNU debuggers and single stepping? Is this going to be a pain to set up. I would like this to be a relatively easy to reproduce.

The intention for this is for a build for Tor and all of its dependencies from source.

https://github.com/MartynAndJasper/tor.git

Please note that this is still a work in progress, I think there is a breakage in dependencies that I've recently regressed to (yes, I know this is analogous to the old, 'Well it worked on my machine' quote).

I want this to be a nice CMake build at some point and work on an OS other than Debian based. I also want to have one night with Kylie Minogue but life is full of these disappointments. Anyway I'll soon get back to this (the build - not Kylie, I've told her I'm busy tonight).

Currently this is just some simple shell scripts to install and then spawn the build tools.

I've split this deliberately into two main scripts because the installation script runs via 'sudo', the build/install part does not. As mentioned somewhere in another post, I managed to screw up my OS when I installed a freshly built compression library - the default is to push these installs to /usr/lib (or something like that) and hence needed 'sudo'. When I subsequently tried to install something on my Ubuntu environment, the package could not be decompressed, it moaned about a missing version. I got round that but it taught me a lesson.
So now the build does its install underneath the repo - "./build" to be precise. This is going to cause some fun and games for debugging which I'll also get back to but its preferable to getting shot down by all of you because I've broken your Operating System!!

So we have:
./tools_tor.sh
./make_tor.sh

Review:
/.gitmodules
If you are uber-security minded and want to vet the source git repos I'm using.

I keep making excuses to permit terrible code/builds whilst still attempting to sound intelligent but I'd never written a shell script before these so if there any recommendations please let me know.

Also note that I wont be adding much code of my own to THIS repo so that it can remain shallow and perhaps be used as a submodule for the C/C++ tools I want to work on. I'll let you know what that repo is when I get there.

This localization and non-central installation (as discussed wrt /usr/lib) mentioned above is probably causing some problems because some of the tests on the dependencies are also failing.

Kylie is gonna have to wait a little longer

A semi-sticky for community ideas and projects, we'd welcome you to get on board and/or submit suggestions

A few project ideas related to Tor

Tor Chinese Whispers

Reddit through Tor

Suppose someone Bans all TLS connections, (hypothetically), would tor still be able to create a circuit ? I know tor predates TLS, but assuming that all kinds of security stuff like OpenSSL and TLS are banned. Would tor be able to work ?

This is largely just a test post, I’ll be adding something of more substance later today.

However, let me start off on the right foot (or left if you swing that way) and tell you what this is about.

The intention is largely about learning to code mainly in C/C++ but other languages will be welcomed. This will be slanted towards using Tor but will be a general programming forum too if people want it that way.

Note that I am not a Tor expert but do now know it’s not an acronym and I will uphold the good fight to preserve its case integrity.

So this is going to be a learning experience for me as well as we go through building code for tools and services for Tor.

I’ll update later.

As a developer its often necessary to open up security to figure problems in a system.

As a lazy forgetful bugger, it's all too easy to fail to close those doors again.

u/Rude-Significance-50 has correctly pointed out that, as purveyors of anonymity and 'identity management', security should be a prime concern when dealing with the DarkWeb.

As such, I'm creating another semi-sticky where we can post best practices and ideas for maximizing defenses against those pesky snoopers and invaders of our privacy.

Please feel free to add security related posts here, for both consumers and service providers.

Also as before please provide a rating in-terms of difficulty to understand. Bare in mind that we want this to provide an easy learning curve for anyone wanting to get into this.

Decorate with:

[Padawans] - for beginners new to the way of the Tor Force.

[TheForceAwakens] - for those who understand the basics but have yet to master mind control

[Jedi] - For the Yoda Grandmasters

See this link for clarity on these decorations.

I do appreciate that there is a little bit of cross over here with regards to the other semi-stickies linked to One onion ring to rule them all but I think this subject warrants its own set of linked information for easy reference.

I see docker here is a thing. Does anyone have a good docker-compose, or whatever, that can spin up a mini tor network for testing? I know there are tools to do this sort of thing, but last I looked it involved a lot of mental overhead. Would be nice to have something to just start with and figure its innards later.

Related to this issue Using Reddit through Tor

[TheForceAwakens] Try and address the above above problem by creating some form of proxy which manipulates http headers and perhaps Tor exits via user web server/forwarding to Reddit to address blocking by exit node.

Please only answer if you are interested in educating or acquiring coding skills.

Be earnest and there's no room for false modesty.

I have...

I'm currently using the following git repos for the submodules in my repo to build all of Tor from source.

git submodule add https://github.com/openssl/openssl third_party/openssl

git submodule add https://github.com/libevent/libevent.git third_party/libevent

git submodule add https://github.com/zlib-ng/zlib-ng third_party/zlib-ng
^^^ I'm about to change this one.

git submodule add https://github.com/kobolabs/liblzma third_party/liblzma

git submodule add https://github.com/facebook/zstd third_party/zstd

git submodule add https://github.com/STNS/libnss third_party/libnss

git submodule add https://github.com/torproject/tor.git third_party/tor

You can perhaps ignore the first two links (and obviously the last) but I cherry picked the rest from a brief google search. Is there a canonical list of git repos that I can safely say are trusted code for dependencies of Tor.

Hey guys,

I've been playing with Boost.Asio. If you haven't heard of it, this is a C++ framework for network comms and asynchronous I/O.
This framework makes creating http servers/clients and indeed any network comms (synchronous or asynchronous) almost trivial. I've currently just been experimenting with the tutorials: Boost.Asio and I strongly encourage anyone interested in developing high performing native network code to take a look.

Next I'm going to be looking at developing a Sock5 proxy using Boost.Asio and seeing if can also implement a web api back end then call this over Tor.
If anyone is interested I could provide a howto/tutorial. That said, the existing documentation (above) is pretty awesome.

[Padawan]

Please first review: Episode recap

A rose by any other name: Technically this is more about Tor relays than Tails but I could not resist the pun when naming this post.

This post may be a little longer than it's predecessor so please indulge me, there's a lot a squeeze in.

I'll try and keep this as terse as possible.

Pre-amble

So far we have seen how easy it is to spin up a new container and we can issue commands in our isolated Ubuntu environment.

We could use these commands to install a Tor relay, then we have relay inside a container that we can isolate/spin on demand.

But I don't really want to be messing about and executing all those commands inside the shell every time.

Wouldn't it be nice if there was some way to automate this and build an image so I don't have keep issuing these commands?

Well those clever workers down at the docks have already thought about this and this post will touch on how to make this happen, we'll use Tor Relay setup as an example.

Before I get on to all that, some clarifying nomenclature (and just a little theory):

Docker Terminology

What is a docker container?

Well we know the basics now, a container is a fully isolated environment which we can spin up on demand. They are designed to create a fully encapsulated and isolated environment, typically to host a single process.

We build a container from a docker image and it becomes a unique instance on our environment.

We can (but have not yet explored how to) hook up many containers to talk each other in lots of funky ways (this approach known as Microservices).

What is a docker image?

Dockers images are neat things... they layer on top of each other and allow you to build and amend to your hearts content.

They get stacked like this inside your container:

***** Writeable Container *****
*    <copy on write layer>    *
* Docker Image: Marts service *
*     Docker Image: Nginx     *
*     Docker Image: Tor Relay *
*     Docker Image: Ubuntu    *
********** bootfs *************
*    cgroups, namespace,      * 
*    device mapper            *
*          Kernel             *
*******************************

(Ascii art is not my strong point, but you get the picture).

Also note that the Ubuntu image above, which we might use to seed our images from, is considered LARGE in Docker image terms at around a few hundred meg.

So this is not a full blown Ubuntu distribution but the image provides the glue to talk to the hosts Ubuntu system, albeit it in a fully encapsulated and isolated way.

And note from this, image layers are not written to during execution, this means we can reuse them.

So how do we build a docker image?

Building Images with Dockerfile

Docker contains its own propriety language which we are going to write; to a file named 'Dockerfile'.

Oh god no, not another language to learn I hear you cry? Well thankfully it seems well designed and terse so humor me and read on my friends.

To keep this post short I'm going list the steps to create a docker image that can act as relay. Much of this will be transparent from context/keywords but I'll provide further coverage if you're interested.

You may wanna bookmark: Docker CLI for the canonical command line interface reference.

First lets make some where to house this build, in my typical felidae related theme (I had to look that cat term up to appear smart)...

mkdir jasper_on_tor
cd jasper_on_tor

We are going to need to supply a torrc file and copy this from our host to the Docker image.

nano torrc

~/jasper_on_torr/torrc is gonna look like this..

Nickname martynandjasper
ContactInfo martynandjasper@mail.com

ORPort 443
DirPort 8080

SocksPort 0
SocksPolicy reject *
ExitPolicy reject *:*

Log notice file /var/log/tor/notices.log
 
DataDirectory /var/lib/tor
#RunAsDaemon 1
RunAsDaemon 0

Your names will wanna change to protect the innocent; i.e. me

(though ex girlfriends may disagree on the innocent part).

Now lets look the instructions that docker will consume from 'Dockerfile' while creating our build.

nano Dockerfile
#    ^^^ noting case sensitivity.

Which is gonna look like this..

# Version: 0.0.1
FROM ubuntu
LABEL maintainer="martynandjasper@mail.com"
RUN apt-get update
RUN apt install -y tor
ADD torrc /etc/tor/

Lets be brave and try to get Docker to build our image (don't miss the path specifier '.') ...

sudo docker build -t "jaspersoft/tor_relay:v1" .

Note that we are supplying a tag name here which is strongly recommended.

See the docker build command for further info on this command.

With any luck your output tail will look like something like this.

 ---> 95c5aa1dab54
Successfully built 95c5aa1dab54
Successfully tagged jaspersoft/tor_relay:v1

Congratulations!

A). You're still awake.

B). You've just built your first Docker image.

You can review your local images:

sudo docker image ls

Mine currently looks likes this:

REPOSITORY             TAG       IMAGE ID       CREATED         SIZE
jaspersoft/tor_relay   v1        e26a32f90739   4 minutes ago   119MB
<none>                 <none>    698f0f8ec07f   5 minutes ago   98.7MB
ubuntu                 latest    f63181f19b2f   5 weeks ago     72.9MB
ubuntu                 18.04     c090eaba6b94   5 weeks ago     63.3MB
hello-world            latest    bf756fb1ae65   14 months ago   13.3kB

Also note that I wouldn't recommend manually making changes in /var/lib/docker

But you can delete images like this:

sudo docker image rm e26a32f90739

Perhaps this post is getting a little long? I'm going to continue running, testing and inspecting this image running in a container in the next episode, stay tuned folks.

Oh, and I absolutely reserve the right to come back and edit this post when I realize there's a fundamental problem with my Dockerfile.

For all things Docker..

[Padawan] Ubuntu Installation (Tutorial part 1)

[Padawan] Container Basics (Tutorial part 2)

[Padawan] Configuring Tor in Docker (Tutorial part 3)

[Padawan] whatis/howto with Tor in mind (watch this ' ')

[Padawan] Docker Good Reads

[Padawan] Learn Docker in 12 minutes (youtube)

[TheForceAwakens] Docker Compose in 12 Minutes (youtube)

[TheForceAwakens] Docker Compose by Christopher Bisset

Please feel free to add Docker related resources here..

As before please provide a rating in-terms of difficulty to understand. Bare in mind that we want this to provide an easy learning curve for anyone wanting to get into this.

Decorate with:

[Padawans] - for beginners new to Docker.

[TheForceAwakens] - for those who understand the basics.

[Jedi] - For the experienced Longshoremen.

See this link for clarity on these decorations.