Pixel Bot Development

So I’m attempting to develop a pixel bot for Diablo 2 or any game for that matter that can run in windowed mode.

Let me start off with sharing my intentions and goals for this project then we can get to the real meat the C# code and development.
The intention is to further grow my skills and anyone else who wants to help in this journey. Also do everything without memory peaking or anything that could possibly get you banned or detected.
This project is NOT intended for commercial use, or the fastest non client bot, or even designed to resell for major bot farms.
My goal is to create a diablo 2 bot that will be able to do these features fully through pixel detection and mouse clicks:

  • Kill Monsters
  • Detect health
  • Mouse and Keyboard output to windowed Diablo 2
  • Looting items
  • Auto pathing maybe?
  • Other features like auto leveling, skilling etc will be more advanced features and come late in the project.

Now for the code and concept:
I’m using VS2015 C# .Net 4.5.2 - Console Application (Since windows forms goes full retard on while loops)

First thing we need to do is get the handle of the diablo window
In the code below we are looking for a Process by the name of “Game” (which is d2’s process name)
If we have more than 1 process we need to throw an error since we don’t know which process to use.
If we have just 1 process we are good and we return the handle: return processes[0].MainWindowHandle; (we use 0 in the process array)
All the other outcomes will return a IntPtr.Zero letting our code know we haven’t found the d2 process.

        public static IntPtr getD2WinHandle()
        {
            Process[] processes = Process.GetProcessesByName("Game");

            if (processes.Count() > 1)
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine("ERROR: More than 1 process detected.");
                Console.ForegroundColor = ConsoleColor.White;
                return IntPtr.Zero;
            }

            if (processes.Count() == 0)
            {
                Console.WriteLine("Open Diablo 2...");
                Thread.Sleep(2000);
                return IntPtr.Zero;
            }

            if (processes.Count() == 1)
            {
                return processes[0].MainWindowHandle;
            }

            return IntPtr.Zero;
        }

Lets now focus on our controls a mouse click:
We need to simulate a real mouse click, each click has a Down and Up event. If we just called the down event then the computer will think we are dragging the mouse so we call both.
Anywhere in our code we can call the MouseClick() function to simulate a click.

        private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
        private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;

        [DllImport("user32.dll")]
        static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, uint dwExtraInfo);

        private static void MouseClick()
        {
            mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);
            mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
        }

Our next control is mouse/cursor movement and position.
We can call the MoveMouseToPosition() function anywhere in our code we just need to define 3 things.
d2 process handle then x & y coordinates, the reason we need the d2handle is so we are referencing coordinates only on that window and not the entire desktop.
Example call of the function: MoveMouseToPosition(D2handle, 150, 30);

        using System.Drawing;    // be sure to add this at the top with the rest of your usings this is needed for Point

        [DllImport("user32.dll")]
        private static extern int GetWindowRect(IntPtr hwnd, out Rectangle rect);
        static private Point ConvertToScreenPixel(Point point, IntPtr hwnd)
        {
            Rectangle rect;

            GetWindowRect(hwnd, out rect);

            Point ret = new Point();

            ret.X = rect.Location.X + point.X;
            ret.Y = rect.Location.Y + point.Y;

            return ret;
        }

        [DllImport("user32.dll")]
        private static extern bool SetCursorPos(int x, int y);

        private static void MoveMouseToPosition(IntPtr hwnd, int x, int y)
        {
            Point point = new Point(0, 0);
            Point windowPoint = ConvertToScreenPixel(point, hwnd);
            Console.WriteLine("Ret: " + ConvertToScreenPixel(point, hwnd));
            SetCursorPos(windowPoint.X + x, windowPoint.Y + y);
        }

Now that we have our d2handle(diablo window) and our controls ready to go we need to detect pixels and color:
Similar to the MousePosition function we are defining our d2handle, x & y coordinates for our GetPixelColor() function which returns a color based on the x&y coords.

        [DllImport("user32.dll")]
        static extern IntPtr GetDC(IntPtr hwnd);

        [DllImport("user32.dll")]
        static extern Int32 ReleaseDC(IntPtr hwnd, IntPtr hdc);

        [DllImport("gdi32.dll")]
        static extern uint GetPixel(IntPtr hdc, int nXPos, int nYPos);

        static public Color GetPixelColor(IntPtr hwnd, int x, int y)
        {
            IntPtr hdc = GetDC(hwnd);
            uint pixel = GetPixel(hdc, x, y);
            ReleaseDC(hwnd, hdc);
            Color color = Color.FromArgb((int)(pixel & 0x000000FF),
                            (int)(pixel & 0x0000FF00) >> 8,
                            (int)(pixel & 0x00FF0000) >> 16);
            return color;
        }

Example of using GetPixelColor():

                    Color pixColor = GetPixelColor(D2handle, 150, 30);
                    Console.WriteLine(pixColor.Name);
                    // pixColor.Name is a string and would look like this: "ff404040"

Here is the testing code I have in its entirety:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Drawing;
using System.Windows.Input;

namespace D2Bot
{
    class Program
    {
        [DllImport("user32.dll")]
        static extern IntPtr GetDC(IntPtr hwnd);

        [DllImport("user32.dll")]
        static extern Int32 ReleaseDC(IntPtr hwnd, IntPtr hdc);

        [DllImport("gdi32.dll")]
        static extern uint GetPixel(IntPtr hdc, int nXPos, int nYPos);

        private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
        private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;

        [DllImport("user32.dll")]
        static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, uint dwExtraInfo);

        [DllImport("user32.dll")]
        private static extern bool SetCursorPos(int x, int y);

        [DllImport("user32.dll")]
        private static extern int GetWindowRect(IntPtr hwnd, out Rectangle rect);
        static private Point ConvertToScreenPixel(Point point, IntPtr hwnd)
        {
            Rectangle rect;

            GetWindowRect(hwnd, out rect);

            Point ret = new Point();

            ret.X = rect.Location.X + point.X;
            ret.Y = rect.Location.Y + point.Y;

            return ret;
        }

        static void Main(string[] args)
        {
            IntPtr D2handle = IntPtr.Zero;
            bool IsD2Open = false;

            while (!IsD2Open)
            {
                D2handle = getD2WinHandle();

                if (D2handle != IntPtr.Zero)
                {
                    Console.ForegroundColor = ConsoleColor.Green;
                    Console.WriteLine("Found D2 Handle: " + D2handle.ToString());
                    Console.ForegroundColor = ConsoleColor.White;
                    IsD2Open = true;
                }
            }

            while(IsD2Open)
            {
                D2handle = getD2WinHandle();

                if (D2handle != IntPtr.Zero)
                {
                    //Bitmap bitmap = new Bitmap(Screen.)

                    Color pixColor = GetPixelColor(D2handle, 150, 30);
                    Console.WriteLine(D2handle.ToString());
                    Console.WriteLine("Color @ 150x150: " + pixColor.Name);

                    switch (pixColor.Name)
                    {
                        case "ff404040":    // Intro
                            Console.WriteLine("Intro.");
                            MoveMouseToPosition(D2handle, 150, 30 + 25);
                            Thread.Sleep(1000);
                            MouseClick();
                            break;
                        case "ff241c10":    // Main Menu
                            Console.WriteLine("Main Menu.");
                            Console.WriteLine("Move Cursor.");
                            MoveMouseToPosition(D2handle, 450, 300 + 25);
                            Thread.Sleep(50);
                            Console.WriteLine("Mouse Click");
                            MouseClick();
                            Thread.Sleep(2000);
                            break;
                        case "ff1c1410":    // Single Player - Char Select
                            Console.WriteLine("Character Select");
                            Thread.Sleep(1000);
                            MoveMouseToPosition(D2handle, 450, 300 + 25);
                            MouseClick();
                            break;
                        default:
                            Console.WriteLine("Color @ 150x150: " + pixColor.Name);
                            break;
                    }
                    //Thread.Sleep(1000);
                    //Console.ReadLine();
                }
                else
                {
                    Console.ForegroundColor = ConsoleColor.Red;
                    Console.WriteLine("D2 has been closed. Reopen D2.");
                    Console.ForegroundColor = ConsoleColor.White;
                    IsD2Open = false;
                }
            }

            Console.ReadLine();
        }

        private static void MoveMouseToPosition(IntPtr hwnd, int x, int y)
        {
            Point point = new Point(0, 0);
            Point windowPoint = ConvertToScreenPixel(point, hwnd);
            Console.WriteLine("Ret: " + ConvertToScreenPixel(point, hwnd));
            SetCursorPos(windowPoint.X + x, windowPoint.Y + y);
        }

        private static void MouseClick()
        {
            mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);
            mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
        }

        public static IntPtr getD2WinHandle()
        {
            Process[] processes = Process.GetProcessesByName("Game");

            if (processes.Count() > 1)
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine("ERROR: More than 1 process detected.");
                Console.ForegroundColor = ConsoleColor.White;
                return IntPtr.Zero;
            }

            if (processes.Count() == 0)
            {
                Console.WriteLine("Open Diablo 2...");
                Thread.Sleep(2000);
                return IntPtr.Zero;
            }

            if (processes.Count() == 1)
            {
                return processes[0].MainWindowHandle;
            }

            return IntPtr.Zero;
        }

        static public Color GetPixelColor(IntPtr hwnd, int x, int y)
        {
            IntPtr hdc = GetDC(hwnd);
            uint pixel = GetPixel(hdc, x, y);
            ReleaseDC(hwnd, hdc);
            Color color = Color.FromArgb((int)(pixel & 0x000000FF),
                            (int)(pixel & 0x0000FF00) >> 8,
                            (int)(pixel & 0x00FF0000) >> 16);
            return color;
        }
    }
}

So if you have followed along until now we currently can do the following:

  • Find the Diablo 2 Window
  • Detect pixel color based off x&y position
  • Move cursor to x&y position
  • Click a mouse

Now the hard part… development comes in, we need to detect enemy monsters, player health and mana and lots of other things but we have the tools to do it. We will focus on this for now and later maybe path finding using game seed etc.
If anyone has any questions, ideas or needs help, more explanation of the code or how to get this working please post!
I will be updating this at the very least weekly to share my latest build and whatever I learn from this.

1 Like

Seems kinda redundant don’t it? Kolbot already does everything your looking for.

But Im not one to rain on an idea, however:
This will only work on your computer (the one picking colors) because different graphic cards will have different hues or tints, not to mention every player personalizes their window (brightness, hue, reds, blues, greens, etc.) so this will only work on your pc.

Is this project something similar with mmBOT ?

Any development is good for d2 botting community.

1 Like

good point, my bad. I may still have a lot of the original files from the mmBot, its written in autoit, and again it will only work on the pc that is doing the color picks. (of course you can always use notes so others can do the edits needed)

Yes this is redundant but its to learn and create something that is open source. Also due to using pixel and no memory reading/hooks this should be fully undetectable unlike others that do use memory & hooks.

It’s going to be like mmBOT but in C# and open source, unfortunately I can’t find any mmBOT source, this would be of great help in development.

1 Like

did you see this one GitHub - ormaaj/d2au3: Diablo II automation framework in autoit , somehow related to mmBOT?

Did you think about a name for your pixel bot?
https://github.com/blizzhackers/D2LoD-files/blob/master/other/Readme.md#pixel-bot

any new updates?

Sorry for the late reply, I hit a wall trying to simulate keystrokes to the Diablo 2 client.

  • The keystrokes are being sent but not registering in D2, for example if I open notepad the Pixel bot has no problem typing a number or letter, however these just don’t register while the D2 window is active.

I’ve sparked an interest in this project again and will post any new updates.

2 Likes

your problem is that some directx apps dont like the keybd_event api calls. Use this. Btw I used your code to implement a full pixel bot. Works but I dont really want to share everything since i sitll use it and dont want it getting out into the wild just yet. Note that the key inputs should be scan codes (Keyboard scancodes: Keyboard scancodes)

using System;
using System.Collections.Generic;
using System.Drawing;
using System.Runtime.InteropServices;
using System.Threading;

namespace IcySorcPindleBot.Helpers
{
public class Input
{
    private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
    private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;
    private const UInt32 MOUSEEVENTF_RIGHTDOWN = 0x0008;
    private const UInt32 MOUSEEVENTF_RIGHTUP = 0x0010;
    public const ushort WM_KEYDOWN = 0x0100;
    public const ushort WM_KEYUP = 0x0101;

    private Random rnd = new Random();

    public static void MoveMouseToPosition(IntPtr hwnd, int x, int y)
    {
        Point point = new Point(0, 0);
        Point windowPoint = ConvertToScreenPixel(point, hwnd);
        SetCursorPos(windowPoint.X + x, windowPoint.Y + y);
    }

    public void RealisticMouseMove(IntPtr hwnd, int x, int y, int distanceScalar, int randomDelta, int sleeptime)
    {
        POINT current;
        GetCursorPos(out current);
        var numBatches = (int)Math.Round(Math.Sqrt((Math.Pow(current.X - x, 2) + Math.Pow(current.Y - y, 2))) / distanceScalar);
        var intervals = GetIntervalPoints(current.X, current.Y, x, y, numBatches, randomDelta);

        foreach (var interval in intervals)
        {
            var randomized = GetRandomizedPoint(interval.X, interval.Y, randomDelta);
            MoveMouseToPosition(hwnd, randomized.X, randomized.Y);
            Thread.Sleep(10);
        }
    }

    public static List<Point> GetIntervalPoints(int x_source, int y_source, int x_dest, int y_dest, int numBatches, int delta)
    {
        double x_curr = x_source;
        double y_curr = y_source;

        double x_step = (double)(x_dest - x_source) / numBatches;
        double y_step = (double)(y_dest - y_source) / numBatches;

        var doubleIntervals = new List<Tuple<double, double>>();

        while (numBatches > 0)
        {
            doubleIntervals.Add(new Tuple<double, double>(x_curr, y_curr));
            x_curr += x_step;
            y_curr += y_step;
            numBatches--;
        }

        var intervals = new List<Point>();

        foreach (var tuple in doubleIntervals)
        {
            intervals.Add(new Point((int)Math.Round(tuple.Item1), (int)Math.Round(tuple.Item2)));
        }

        return intervals;
    }

    public static POINT GetCursorPosition()
    {
        POINT lpPoint;
        GetCursorPos(out lpPoint);
        // NOTE: If you need error handling
        // bool success = GetCursorPos(out lpPoint);
        // if (!success)

        return lpPoint;
    }

    public Point GetRandomizedPoint(int x, int y, int delta)
    {
        return new Point(rnd.Next(x - delta, x + delta), rnd.Next(y - delta, y + delta));
    }

    public static void LeftMouseClick()
    {
        mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);
        mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
    }

    public static void RightMouseClick()
    {
        mouse_event(MOUSEEVENTF_RIGHTDOWN, 0, 0, 0, 0);
        mouse_event(MOUSEEVENTF_RIGHTUP, 0, 0, 0, 0);
    }

    public static void MouseClickDown()
    {
        mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);
    }

    public static void MouseClickUp()
    {
        mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
    }

    public static void KeyboardPress(IntPtr D2handle, uint scan)
    {
        SetActiveWindow(D2handle);

        SendKey(scan, false, InputType.Keyboard);
        SendKey(scan, true, InputType.Keyboard);
    }

    /// <summary>
    /// Sends a directx key.
    /// http://www.gamespp.com/directx/directInputKeyboardScanCodes.html
    /// </summary>
    /// <param name="key"></param>
    /// <param name="KeyUp"></param>
    /// <param name="inputType"></param>
    public static void SendKey(uint key, bool KeyUp, InputType inputType)
    {
        uint flagtosend;
        if (KeyUp)
        {
            flagtosend = (uint)(KeyEventF.KeyUp | KeyEventF.Scancode);
        }
        else
        {
            flagtosend = (uint)(KeyEventF.KeyDown | KeyEventF.Scancode);
        }

        InputStruct[] inputs =
        {
        new InputStruct
        {
            type = (int) inputType,
            u = new InputUnion
            {
                ki = new KeyboardInput
                {
                    wVk = 0,
                    wScan = (ushort) key,
                    dwFlags = flagtosend,
                    dwExtraInfo = GetMessageExtraInfo()
                }
            }
        }
    };

        SendInput((uint)inputs.Length, inputs, Marshal.SizeOf(typeof(InputStruct)));
    }

    static private Point ConvertToScreenPixel(Point point, IntPtr hwnd)
    {
        Rectangle rect;

        GetWindowRect(hwnd, out rect);

        Point ret = new Point();

        ret.X = rect.Location.X + point.X;
        ret.Y = rect.Location.Y + point.Y;

        return ret;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct POINT
    {
        public int X;
        public int Y;

        public static implicit operator Point(POINT point)
        {
            return new Point(point.X, point.Y);
        }

        public override string ToString()
        {
            return "(" + X.ToString() + ", " + Y.ToString() + ")";
        }
    }

    [Flags]
    public enum KeyEventF
    {
        KeyDown = 0x0000,
        ExtendedKey = 0x0001,
        KeyUp = 0x0002,
        Unicode = 0x0004,
        Scancode = 0x0008,
    }

    [Flags]
    public enum InputType
    {
        Mouse = 0,
        Keyboard = 1,
        Hardware = 2
    }

    public struct InputStruct
    {
        public int type;
        public InputUnion u;
    }

    [StructLayout(LayoutKind.Explicit)]
    public struct InputUnion
    {
        [FieldOffset(0)] public readonly MouseInput mi;
        [FieldOffset(0)] public KeyboardInput ki;
        [FieldOffset(0)] public readonly HardwareInput hi;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct MouseInput
    {
        public readonly int dx;
        public readonly int dy;
        public readonly uint mouseData;
        public readonly uint dwFlags;
        public readonly uint time;
        public readonly IntPtr dwExtraInfo;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct KeyboardInput
    {
        public ushort wVk;
        public ushort wScan;
        public uint dwFlags;
        public readonly uint time;
        public IntPtr dwExtraInfo;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct HardwareInput
    {
        public readonly uint uMsg;
        public readonly ushort wParamL;
        public readonly ushort wParamH;
    }

    [DllImport("user32.dll", SetLastError = true)]
    private static extern uint SendInput(uint nInputs, InputStruct[] pInputs, int cbSize);

    [DllImport("user32.dll")]
    private static extern IntPtr GetMessageExtraInfo();

    [DllImport("user32.dll")]
    private static extern int GetWindowRect(IntPtr hwnd, out Rectangle rect);

    [DllImport("user32.dll")]
    public static extern bool GetCursorPos(out POINT lpPoint);

    [DllImport("user32.dll")]
    private static extern bool SetCursorPos(int x, int y);

    [DllImport("user32.dll")]
    static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, uint dwExtraInfo);

    [DllImport("user32.dll")]
    static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, int dwExtraInfo);

    [DllImport("user32.dll")]
    public static extern IntPtr SetActiveWindow(IntPtr hWnd);

    [DllImport("user32.dll")] //sends a windows message to the specified window
    public static extern int SendMessage(IntPtr hWnd, int Msg, uint wParam, int lParam);
}

}

also important to note that you probably want to strip off the llkhf and llmhf injected flags from your input. Here is a good repo for how to do that. LowLevelMouseHook-Example/LLMHREmover at master · Sadulisten/LowLevelMouseHook-Example · GitHub

You can clone the repo and just run this after starting d2 and before running your bot

A post was split to a new topic: D2r pixel bot

“Pixel bot” I’d rather call this kind of bot “bot that uses human interface”. So, IMHO, the point of these bots are either to:

  • learn programming related to games
  • try AI / ML bots (i.e. bots that use vision, language maybe reinforcement learning, but it’s painful to implement a proper training environment)

I’ve tried something lately for the later GitHub - MephisTools/diablo2-multimodal: Multimodal deep learning experiments on Diablo 2 data
The idea would be to make bots programmable in natural language that uses the same input/output than humans (humans can’t read the RAM or see the network packets etc.):

  • “get to level 99”
  • “start game, kill countess, restart”

Probably in the path of https://cliport.github.io/