CPU Mining cryptocurrencies using Apache Spark without any external dependencies.
Using CPU Miner from https://github.com/pooler/cpuminer in an Apache Spark job that starts tasks on a Yarn/Mesos cluster. Each cluster executor starts an instance of CPU Miner using the provided mining options.
Get yourself an account in any mining pool that supports CPU mining :
| Currency | Mining pools |
|---|---|
| Bitcoin | https://en.bitcoin.it/wiki/Comparison_of_mining_pools |
| Litecoin | https://www.litecoinpool.org/ |
| Dogecoin | https://aikapool.com/doge/index.php?page=login |
Keep an eye on https://coinmarketcap.com/ and look for new coins that do not require special mining equipments.
Package the project into a single jar (requires SBT), in the project root run :
sbt package
You'll get a spark-yarn-miner_2.11-*.jar jar.
Next, use spark-submit to submit your job to the cluster :
spark-submit --master yarn --class fr.marouni.spark.coins.SparkYarnMiner spark-yarn-miner_2.11-0.1.0-SNAPSHOT.jar --url XXX --username YYY --worker-id WWW --password UUUU
Where :
--url: Your mining pool URL (check mining pool docs). Example : stratum+tcp://stratum.aikapool.com:7915--username: Mining pool account username--worker-id: Mining pool worker id (check mining pool docs)--password: Mining pool account password
Tune your mining with the following spark-submit options :
spark-submit --conf spark.executor.instances=10on a 10 nodes clusterspark-submit --conf spark.executor.cores=6on a cluster with 6 cores per node
Recommended tuning : 1 executor per cluster node with maximum allowed number of cores per executor.