Developer Testimonials

How to be a successful remote software engineer

This blog post covers how a developer can be recognized, find a remote job, and be successful working remotely. However, it’s important to understand what is in it for organizations hiring remote developers.

*Full Disclaimer: All the views expressed in the blog are solely my personal views and biased based on my personal experience. The best-practices, technologies, or benefits listed are no silver bullets. The article is focused on engineers in the web development space. 

Remote working is not a new concept but working remotely is gaining popularity during these times. Many organizations are forced to rethink how they work. Covid-19, has impacted every person in the world, but with the challenges the pandemic has created comes opportunity.

There has never been a better time to work remotely, especially for engineers. Many organizations have now turned remote-friendly, some have also started hiring developers from regions unheard of – the reason? Untapped potential!!

This blog post covers how a developer can be recognized, find a remote job, and be successful working remotely. However, it’s important to understand what is in it for organizations hiring remote developers.

Why organizations should hire remote developers

It would be unfair to say that organizations do NOT want to hire remote developers as they are bound by government laws which prevent them from going beyond their country to hire a person on their payroll. It’s only possible for large corporations.

Let’s first understand the benefits of hiring remote engineers. As an organization you can:

  • Hire engineers who have untapped potential – these engineers have high productivity and are eager to learn.
  • Hire engineers from countries with lower GDP – allows you to pay people less than you would if you hire local talent.
  • Add diversity to your company culture – allows sharing different ideas and perspectives that you did not have before.
  • Become a 24×7 company – by hiring engineers in varied timezones you can move faster and support customers globally.

Now that you understand why a company wants to hire you, let’s discuss why a developer would want to work remotely.

Why engineers should consider remote jobs

There are many benefits (and few drawbacks) of working as a remote engineer. 

  • Choose your hours – Since you work in a different timezone, you can choose the hours you want to work, though it’s important to have some overlap. (more on that later)
  • Work with people with diverse backgrounds – there is a different thing about people who are well-traveled, right? Why is that?
  • Get paid more than your peers – you can only earn what your industry pays you, what if you changed the local industry? 😉
  • Choose where you work– Home, Coworking office, Coffee Shop? It’s recommended though you have a consistent setup (again, later!)
  • Better work-life balance – Save time traveling, get more time off (remote organizations are usually flexible), be with your loved ones often.
  • Choose your own technologies – though it helps get better jobs depending on the tech you work with, organizations are looking for the skills you’ve developed to help them identify what you’d work on.
  • Immense growth – working with people globally brings a lot of different perspectives allowing you to 10x your growth.

Why would organizations consider you

We have established that organizations want remote engineers, now let’s look at why an organization would consider you? What do you need that makes an organization believe you are remote-friendly?

Open Source Contributions
Organizations want to look at the work you have done. 

  • It increases the credibility of your work
  • It shows that you love writing code
  • It gives them a glimpse into the code you write

Remote-friendly technologies
If you are looking to join a startup, most likely they use technologies that are popular right now. Having experience in current tech is a great way to get noticed. Some of these technologies are (but not limited to):

  • Javascript (Node and React)
  • GraphQL
  • Python (Django)
  • Kubernetes and other cloud devops experience is a huge plus

It usually helps to be able to work on both backend and frontend (Full-Stack), since it’s crucial to be self-driven in a remote environment.

Solid previous experience and profile
Organizations love when they find a person who is a great problem solver. Working on multiple projects and industries, at different roles, are usually indications that you will do well in their company. Companies will also check your Linkedin profile to understand you better. Having an updated profile and strong recommendations from previous employment can go a long way towards helping you find the right remote job.

Attitude
I saved the most important one for the last. Companies hire for attitude rather than skill. Skill can be learned, but attitude takes a long time to correct. Having the right attitude is the only way to get good remote jobs. 

So what do I mean by having the right attitude? 

Display a willingness to learn more about their company, show a genuine interest in the company’s industry and what it cares about. Read the company’s vision, its core values, culture, and apply only if these attributes excite you. It’s essential that you’re a quick learner so that you can developed the required skills to perform at the company.

How to find remote jobs

So, if you have what it takes to be a good remote engineer, the question is, how do you find a remote job that you love?

Apply to a company’s remote jobs (via portals or company website)

If you do a quick search on Google, you will see many platforms like WeWorkRemotely, remote.co, and others.

You can start by looking at the skill you want to target and applying on the posts (make sure you research the company before applying). You need to have a great cover letter. Cover letters are a great way to express why you are the best person for the job they posted. A strong cover letter makes you stand out as companies receive 100s of applications.

Which application do you think they are most likely to open first? The one with the cover letter! You can also search for companies which are remote-friendly and apply directly via their websites.

Freelance

Freelancing is also another way to get jobs. Freelance positions offer more flexibility and let you have a better work-life balance. Freelance work also gives you the chance to choose your hourly rate, but you may sacrifice job security, and you might also waste time hunting for your next gig.

Platforms like Turing, guru, Upwork, and freelancers are good places to find remote gigs.

Personal Connections

Twitter is a great place to build relationships with other fellow developers. These connections will help you find your next job. 60% of organizations hire people that are referred by the people already working in their companies. This means the more people you know in the industry, the better chance you will have to get a good job.

Turing.com 

Turing is a unique platform that bridges the gap between a freelance platform and a job portal. It is truly focused on the developer’s well-being, growth, and tools to be successful in working remotely.

Turing is different because:

  1. You do not have to hunt for jobs – Turing will understand your goals and find you a job that you want. 
  2. You get long term work – You will work with a real company as their team member. You get the benefits of the company you are working for under turing. 
  3. You still get the flexibility as you choose your own hours and your rate.
  4. Turing pays you on time – you do not have to follow up with your clients to get paid or depend on a rating system to get jobs. 
  5. Turing handles issues that may arise between you and the client. 

Turing gives developers peace of mind by allowing them to focus on their skills and their job instead of spending time doing administrative work that reduces their productivity.

Working Remotely

Getting a job is only the first step. There is a lot more that you need to do to be successful at your job. 

Communication
Being an effective communicator is the key to being successful at a remote job. Working remotely means you need to make extra efforts to communicate with your manager.

  • Have regular check-ins with your manager (weekly as well as monthly)
  • Have at least 3 hours of time overlap between yours and your team’s work hours. 
  • Make sure you and your manager(and your team) are always on the same page, and that expectations are clearly understood. 

Turing.com actually does a great job improving your communication with your manager. 

Self-driven
You need to be self-driven. The more you have to depend upon another person on the team, the more difficult it will get to be productive in your job. It certainly helps if you are a full-stack engineer, as this allows you to do both the frontend and the backend by yourself – if it is not possible then you must try to separate (but not isolate) your responsibilities.
The more time overlap you have with your team, the more flexible you can be with respect to separating your work responsibilities.

Setup
Having a decent office and workstation setup is very important. You cannot be productive at your work if you have “pebbles” on the race track you are trying to win.

  • Make sure you have a good (and consistent) place to work
  • Your environment should be distraction-free
  • Good camera and microphones to have calls with your team. 
  • A fast computer that can handle your daily workload

Trust
Remote teams are happy and do more if they trust each other. Here is a great article that would do justice to explaining how important trust is in a workplace. https://blog.doist.com/trust-remote-workplace/

With this, I wish you luck finding a great remote company to work at. It can be hard, but rewarding. I trust that turing.com can help you find the next job that you love. 🙂

By October 27, 2020
Developers Corner

Onboarding to a new team as an engineering leader

In this post, Jean Hsu of Range shares some guiding principles and practices that have been helpful to her in navigating this onboarding process as an engineering leader.

Editor’s Note: This is a guest post from Jean Hsu of Range.

I recently joined Range as their new VP of Engineering. Over the last few weeks, I’ve ended many days full of meetings feeling energized — grateful to work with this incredible group of humans. And to be honest, I’ve also ended days feeling depleted — feeling a bit bashful about basic questions and overwhelmed by all that I don’t know.

Although I’ve previously onboarded at big companies like Google and smaller startups like Medium and built onboarding programs for engineering teams, this is the first time I’ve onboarded to a team in over eight years. It’s also the first time I’ve been onboarded to a team while everyone is working remotely, not to mention in the middle of a pandemic, while my kids are distance learning from home! With those remote constraints and personal time constraints in mind, I wanted to be particularly intentional about how I spent the first few weeks.

In this post, I’ll share some guiding principles and practices that have been helpful to me in navigating this onboarding process. 

Use Structured Questions to Get to Know Individuals and the Team
One-on-ones are foundational in getting to know people as individuals. You will want to schedule recurring one-on-one meetings with people you work closely with — whether that’s direct reports, cross-functional leads, or your manager.

In your first or second one-on-ones with the team, ask a set of structured questions to guide the conversation. You can give people a heads-up that you’ll be doing so, so they know it won’t be the norm for all one-on-ones. These are the questions I asked everyone on the engineering team:

  • What’s going well at Range?
  • What’s been frustrating, or could be better?
  • If you could have your way, what one thing would you change?
  • What do you want to get out of your time at Range?
  • What support can the team or I provide?

Think of these questions as a broad invitation to share whatever they feel is important. There are few enough that there’s plenty of time to dig into the responses in a 45 minute or hour-long time frame. Delve deeper into each with open-ended follow-up questions like “What else?” and “Can you tell me more about that?”

Without a clear intention, over time, one-on-ones can settle into status updates or pleasant-but-not-too-meaningful chitchat.  By bringing up these topics at the start of a new work relationship, you let the other person know that the one-on-one space is one where these topics can be discussed. One-on-ones are the venue where you want to hear what’s going well, learn about any frustrations, discuss areas ripe for change, what your direct reports want professionally, and what support they need. 

Lean into Your Beginner’s Mind
When you’ve been on a team for years, working day-in and day-out in the same codebase and same team, you acclimate to small changes around you, like slowly increasing build times or that weekly meeting that doesn’t seem to have an agenda. Blindspots emerge that slow the team down significantly.

When you’re the newcomer to a team, you’re the only one with entirely fresh eyes. Take notes on what you notice. Are there product features that seem particularly delightful to you? Do you find any processes that feel needlessly painful? What about obvious gaps that feel important to fill?

It’s easy to tell yourself, “Oh, I’m new, so I’m sure they have a good reason for that. I’ll just keep my mouth shut and see if it all makes more sense in a few months.” It’s tempting not to want to rock the boat and not be the new engineering leader associated with complaints. Quite reasonably, you don’t want to be the person who chimes in at every meeting with, “Well, at Google, we did XYZ.”

To get around being the “problem messenger,” get buy-in upfront from other leaders with whom you work closely. Talk to them about what gaps you can fill in the leadership team, and discuss processes for you to leverage your “Beginner’s Mind” in this critical period to share observations and insights.

Absorb Information, and Let Go of Your Need to Know Everything
At Medium, the previous tech company I worked at, I joined before there was a Medium. I was there through the nascent ideation process, building out of the initial product and every single product iteration after that. 

At Range, I don’t have that in-depth knowledge to lean on.
Suppose you are, like me, joining a company as an engineering leader. In that case, you may be trying to absorb everything you can about the team, the individuals, the processes, the codebase, and the product. Piece together what you can — have conversations with engineers, designers, product people, sales, and marketing. Read relevant docs, and learn from the expertise others have on the team.

And know that you don’t need to have that full historical context to fill your role effectively. I also remember times at Medium when I had no context at all. Once, I helped DevOps scope out a plan for thwarting DDOS attacks, even though I had no prior meaningful knowledge concerning this issue. I scoped out and executed a successful multi-month API project, with little context as well. 

So absorb what you can to get up to speed and let go of your need to know everything. Ask questions when you have them, and ask for help when you get stuck. Trust that you’ll tap into your team’s expertise to get the information you need to lead teams and projects. 

Define Your Role
As you settle in and start to get a feel for the team’s needs, take some time to take a step back and define your role. It can be easy as the new person to help out everywhere as needed, but take the time to think about what you want the position to be — what do you want to be doing six months or a year into your job?

There will be parts of your role that are more concrete and non-negotiable, but engineering leadership roles often have a lot of room to choose your adventure. 

I love to write, so part of my role definition includes external-facing influence through writing blog posts and helping with other content for the product. Someone else may want to carve out time for regularly preparing and delivering talks or play a meaningful role in defining and iterating on team processes. 

When I’ve taken the time to clarify my role in this way, it helps to contextualize the day-to-day tasks and feel less scattered and reactive. It’s analogous to taking the time to define and communicate a team’s North Star and top priorities. Even if individuals are working on varied tasks, it’s essential to know how it ratchets up to the team’s focus — and that also helps individuals be mindful of when their work doesn’t contribute clearly to the team’s priorities. Similarly, taking the time to define my ideal role gives me clear intention and direction — so rather than feeling scattered or overwhelmed, I can see how the disparate parts of my job add up towards a role I aspire to fill.

Joining a new team as an engineering leader can be exhilarating, daunting, joyful, and overwhelming — sometimes all in the same day! You may be pulled in all directions before you even settle in. While you’re getting up-to-speed, remember to keep just a few priorities top-of-mind and communicate them clearly (even if they change every few weeks). I hope these principles and practices help you navigate this transition. 

About Jean:
Jean
 Hsu is the Vice President of Engineering at Range. Prior to Range, she built product and engineering teams at Google, Pulse, and Medium, and co-founded Co Leadership, a leadership development company for engineers and other tech leaders. She’s also a co-actively trained coach and has coached many engineers, tech leads, managers, PMs, VPs of Engineering, and CTOs. She loves to play ultimate frisbee (though not during pandemics), and lives in Berkeley with her partner and two kids.

About Range:
Crafting new ways for organizations, teams, and individuals to unlock their full potential

The team at Range believes that healthy companies aren’t simply better places to work, but do better work and will ultimately be more successful. But that’s easier said than done — it often seems the more humans an organization adds, the less human it becomes.

We think this can (must!) be fixed, and that by putting (awesome) team success software into people’s hands, they can build wellbeing, awareness, and performance into the fabric of work.

By October 26, 2020
Turing News

Turing’s Boundaryless Product Event – Fall Edition

Turing’s Boundaryless Product Event – Fall Edition Save the Date: Thursday, October 15th, 11 AM – 2 PM PT   (Join and/or RSVP here) This Thursday, Turing not only introduces you to exciting new product innovations, but also brings together top remote-work experts and advocates in the inaugural Turing Boundaryless Product Event. Over the course of… View Article

Turing’s Boundaryless Product Event – Fall Edition

Save the Date: Thursday, October 15th, 11 AM – 2 PM PT   (Join and/or RSVP here)

This Thursday, Turing not only introduces you to exciting new product innovations, but also brings together top remote-work experts and advocates in the inaugural Turing Boundaryless Product Event. Over the course of the virtual event, learn how remote-distributed teams can turbo-charge your development, increase your runway, reduce fixed costs, and make your company more attractive to investors. Additionally, the event will allow you to:

  • Hear from scaling experts that have built world-leading products with remote teams. 
  • See the latest product innovations from Turing, the company building AWS for talent. 
  • Learn about the Future of Work with Ting Cai, formerly of Microsoft and now, Senior Director at Google

Please join Turing CEO, Jonathan Siddharth as well as special guests:

Registration and access to the event is free of charge — please RSVP and/or join us here.


Schedule of events:

Turing Product Launch Demo (11am – 11:45am) by Turing CEO and Co-Founder Jonathan Siddharth

Have a first look at Turing’s exciting new product innovations. See how hiring and managing Silicon Valley-caliber engineers can now be faster and easier than ever before.

Panel Discussion: Scaling with Remote Teams (11:45am – 12:25pm)  — by Prakash Gupta with, Dennis Payonk, and Richard Hong

Hear from current customers about their experience building engineering teams on top of Turing. Learn remote work best practices, how to avoid common mistakes, and more.

Fireside Chat with Ting Cai of Google, Ashu Garg of Foundation Capital, and Turing Co-Founder and CEO, Jonathan Siddharth on the Future of Remote Work (12:30pm – 1:30pm)

Join Jonathan Siddharth, Turing’s CEO and Co-Founder, Ashu Garg of Foundation Capital, and Ting Cai, formerly of Microsoft and now, Senior Director at Google, in a lively discussion about the future of work, moderated by TechCrunch reporter, Ingrid Lunden.

Turing Developer Stories — Building Great Products while Changing Lives (1:30pm – 2pm)

See first-hand how Turing changes the course of developers’ lives worldwide while helping customers scale engineering teams quickly, even amid a global pandemic.

Register for free here

Jonathan Siddarth, CEO & Co-Founder, Turing

Prakash Gupta, Founding CRO, Turing

Ting Cai, Senior Director at Google

Ashu Garg, GP, Foundation Capital Ashu Garg, GP, Foundation Capital

Richard Hong, Co-Founder at Pangaea

Dennis Payonk, VP of Engineering at VillageMD

By October 12, 2020
Developers Corner

PART 2: UNDERSTANDING MySQL CLIENT / SERVER PROTOCOL USING PYTHON AND WIRESHARK

In this article we’ll learn how to write our own native MySQL client from scratch using no connector or external libraries.

In the previous article we researched MySQL Client / Server Protocol using WireShark. Now lets start to write our code in python to simulate MySQL native client. Final codes are here: Github repo

First of all we have to create MYSQL_PACKAGE class. MYSQL_PACKAGE class is the parent of all other package classes (HANDSHAKE_PACKAGE, LOGIN_PACKAGE, OK_PACKAGE and etc.)

It accepts resp parameter on initialization. Resp is the binary response received from the server in bytesarray type. One of the important and interesting method of this class is next method.

Method next reads a portion of the bytes from the binary response. When we call this method, it reads some portion of bytes and puts a pointer to the last position where reading ended (changes a value of self.start and self.end properties). When we call this method again, it starts to read bytes at the point it last stopped.
Method next accepts five parameters: length, type, byteorder, signed, and freeze. If freeze is True it reads some portion of bytes from the binary response but does not change pointer position. Otherwise it reads a portion of bytes with given length and changes the position of pointer. If length is None then method reads bytes until the end of response bytesarray. Parameter type can be int, str, and hex data types. Method next converts a portion of bytes into the appropriate datatype according to the value of type parameter.
Parameter byteorder determines the conversion of bytes to integer type. It is up to the architecture of your computer. If your machine is big-endian, then it stores bytes in memory from the big address to the little. If your machine is little-endian, then it stores bytes in memory from the little address to the big. Thats why we have to know the exact type of our architecture to be able to convert bytes to integer correctly. In my case, it is little-endian, that’s why i’ve set the default value of byteorder parameter to “little”.
Parameter signed is also used in conversion of bytes to integer. We tell the function to consider each integer as unsigned or signed.
A second interesting method of this class is encrypt_password. This method encrypts a password with the given algorithm.

This method accepts two parameters: salt and password. Parameter salt is the concatenation of two salt1 and salt2 strings from the Greeting Packet received from the server. And parameter password is the password string of mysql user.
In the official documentation password encryption algorithm is:
password_encrypt_algorithm
Here “20-bytes random data from server” is concatenation of salt1 and salt2 from the Greeting Packet received from server. To remember what the greeting packet is look at the previous article
Now I want to explain the encrypt_password method line by line.
bytes1 = sha1(password.encode(“utf-8”)).digest()
We are converting password string to bytes, then encrypting it with sha1 function and assigning to bytes1 variable. It is equal to this part of algorithm:
password_encrypt_algorithm1
Then we are converting salt string into bytes and assigning to the concat1 variable.
concat1 = salt.encode(‘utf-8’)
password_encrypt_algorithm5
Third line of the method is:
concat2 = sha1(sha1(password.encode(“utf-8”)).digest()).digest()
password_encrypt_algorithm2
Here we are double-encrypting password string with sha1 function and assign it to the concat2 string.
Now we have two concat1 and concat2 variables. We have to concatenate them into one byte array:
bytes2 = bytearray()
bytes2.extend(concat1)
bytes2.extend(concat2)
password_encrypt_algorithm6
Then we have to encrypt concatenated bytes with sha1 function and assign to the bytes2 variable.
bytes2 = sha1(bytes2).digest()
password_encrypt_algorithm3
So we have two variables with encrypted bytes: bytes1 and bytes2. Now we have to do bitwise XOR operation between these variables and return the obtained hash.
hash=bytearray(x ^ y for x, y in zip(bytes1, bytes2))
return hash
password_encrypt_algorithm4

CLASSES FOR DATATYPES

In the previous article we’ve learned about Int and String data types of MySQL Client / Server protocol. Now we need some classes to be able to read fields from received packets.

INT CLASS

Int class implements INT data type of MySQL Client / Server protocol. It accepts package parameter on initialization. Parameter package should be the instance of any package class inherited from MYSQL_PACKAGE class. Method next detects the type of integer (int<fix> or int<lenenc> (see previous article) and calls the next method of package object to read the byte portion of received response.

STR CLASS

Str class implements STRING data type of MySQL Client / Server protocol. It accepts package parameter on initialization. Parameter package should be the instance of any package class inherited from MYSQL_PACKAGE class. Method next detects the type of String (String<fix>, String<Var>, String<NULL>, String<EOF> or String<lenenc>. See previous article) and calls the next method of package object to read the byte portion of received response.

HANDSHAKE_PACKAGE CLASS

HANDSHAKE_PACKAGE class is used for parsing the Greeting Packet received from server. It is inherited from MYSQL_PACKAGE class and accepts resp parameter on initialization. Parameter resp is the Greeting Packet response in bytes type recieved from the server.

Method parse reading fields from the response using Int and Str classes and puts them into a dictionary and returns.

LOGIN_PACKAGE CLASS

This class is used for create Login Request packet.

This class accepts handshake parameter on initialization. Parameter handshake should be the instance of HANDSHAKE_PACKAGE class. In the __init__ method we call the parse method of handshake object and get all fields of the Greeting Packet received from the server.
Method create_package prepares the login request package to be able to send to the server for authentication. Accepts user, password and packet_number parameters.

OK_PACKAGE & ERR_PACKAGE CLASSES

OK package and ERR package are the response package of server after authentication or after sending query to server on command phase.

MYSQL CLASS

MYSQL class is the wrapper class which creates TCP connection with server, sends and receives packages from server using above classes.

I think everything is clear in this class. I’ve defined __enter__ and __exit__ to be able to use this class with “with” statement to automatically close TCP connection. In __enter__ method i’m creating TCP connection over socket. And in __exit__ method i’m closing created connection. This class accepts host, port, user and password parameters on initialization.
In the connect method we receive greeting packet from server:
resp = self.client.recv(65536)
return HANDSHAKE_PACKAGE(resp)
In the login method we create Login request package using LOGIN_PACKAGE and HANDSHAKE_PACKAGE classes and sends to the server and gets OK or ERR packages.
That’s all. We’ve implemented the connection phase. To avoid making this article too long I will not explain the command phase. Because the command phase is easier than the connection phase. You can research it yourself with the knowledge you’ve accumulated from this and previous articles.
Demo Video:

By October 9, 2020
Developers Corner

UNDERSTANDING MySQL CLIENT / SERVER PROTOCOL USING PYTHON AND WIRESHARK – PART 1

MySQL Client / Server protocol is used in many areas. For example: MySQL Connectors like ConnectorC, ConnectorJ and etc. MySQL proxy Between master and slave What is MySQL Client / Server protocol? MySQL Client / Server protocol is accepted conventions (rules). Through these rules client and server “talks” and understand each other. Client connects to… View Article

MySQL Client / Server protocol is used in many areas. For example:

  • MySQL Connectors like ConnectorC, ConnectorJ and etc.
  • MySQL proxy
  • Between master and slave

What is MySQL Client / Server protocol?

MySQL Client / Server protocol is accepted conventions (rules). Through these rules client and server “talks” and understand each other. Client connects to server through TCP connection with special socket, sends to server special packets and accepts them from server. There are two phases of this connection:

  • Connection phase
  • Command phase

Next illustration describes phases:

STRUCTURE OF PACKETS

Each packet consists of valuable data types. Maximum length of each packet can be 16MB. If the length of packet is more than 16MB, then it is separated into several chunks (16MB). First of all let’s see the protocol data types. MySQL Client / Server protocol has two data types:

  • Integer types
  • String types

(See the official documentation: https://dev.mysql.com/doc/internals/en/basic-types.html)

INTEGER TYPES

Integer types also separates into two section:

  • Fixed length integer types
  • Length-encoded integer types

Fixed length integer type consumes 1, 2, 3, 4, 6 or 8 bytes. For example if we want to describe number 2 in int<3> data type then we can write it like this in hex format: 02 00 00. Or if we want to describe number 2 in int<2> then we can write it like this in hex format: 02 00

Length-encoded integer types consumes 1, 3, 4 or 9 bytes. Before length-encoded integer types comes 1 byte. To detect the length of integer we have to check that first byte.

  • If the first byte is less than 0xfb ( < 251 ) then next one byte is valuable (it is stored as a 1-byte integer)
  • If the first byte is equal to 0xfc ( == 252 ) then it is stored as a 2-byte integer
  • If the first byte is equal to 0xfd ( == 253 ) then it is stored as a 3-byte integer
  • If the first byte is equal to 0xfe ( == 254 ) then it is stored as a 8-byte integer

But if the first byte is equal to 0xfb there is no need to read next bytes, it is equal to the NULL value of MySQL, and if equal to 0xff it means that it is undefined.

For example to convert fd 03 00 00 … into normal integer we have to read first byte and it is 0xfd. According to the above rules we have to read next 3 bytes and convert it into normal integer, and its value is 2 in decimal number system. So value of length-encoded integer data type is 2.

STRING TYPES

String types also separates into several sections.

  • String – Fixed-length string types. They have a known, hardcoded length
  • String – Null terminated string types. These strings end with 0x00 byte
  • String – Variable length string types. Before such strings comes fixed-length integer type. According to that integer we can calculate actual length of string
  • String – Length-encoded string types. Before such strings comes length-encoded integer type. According to that integer we can calculate actual length of string
  • String – If a string is the last component of a packet, its length can be calculated from the overall packet length minus the current position

SNIFF WITH WIRESHARK

Let’s start wireshark to sniff the network, filter MySQL packets by ip (in my case server ip is 54.235.111.67). Then let’s try to connect to MySQL server by MySQL native client on our local machine.

>> mysql -u[username] -p[password] -h[host ip] -P3306

As you can see after TCP connection to the server we several MySQL packets from the server. First of them is greeting packet.

picture1

Let’s dig into this packet and describe each field.

First 3 bytes are packet length:

picture2

Next 1 byte is packet number:

picture3

Rest of bytes are payload of Greeting packet of MySQL Client / Server protocol

picture4

Let’s describe each field of greeting packet.

  • Protocol number – Int<1>
  • Server version – String
  • Thread id – Int<4>
  • Salt1 – String
  • Server capabilities – Int<2>
  • Server language – Int<1>
  • Server Status – Int<2>
  • Extended Server Capabilities – Int<2>
  • Authentication plugin length – Int<1>
  • Reserved bytes – 10 bytes
  • Salt2 – String
  • Authentication plugin string – String

Server language is integer, next table will help us to pick appropriate language by integer value:

In my case server language is 0x08 (in decimal number system it is 8 also). From above table we can see that equivalent of 8 is latin1_swedish_ci. Now we know that default language of server is latin1_swedish_ci.

Server capabilities and server status are also integers. But reading each BIT of these integers we can know about server capabilities and status. Next illustration describes server capability and status bits:

Using greeting packet client prepares Login Request Packet to send to the server for authentication. Now let’s research login request packet.

picture5

  • First 3 bytes describes payload length
  • Next 1 byte is packet number
  • Client capabilities – Int<2> / Same as Server capabilities
  • Extended client capabilities – Int<2> / Same as Server extended capabilities
  • Max packet – Int<4> / describes the maximum length of packet
  • Charset – Int<1> / in my case it is 0x21 (in decimal number system is 33), from the table we can see that it is utf8_general_ci. We set server’s default charset from latin1_swedish_ci to utf8_general_ci
  • Username – String
  • Password – String
  • Client Auth Plugin string – String

As you can see password is encrypted. To encrypt a password we will use sha1, md5 algorithms, also salt1 and salt2 strings from previous Greeting Packet sent from server.

Then we get OK packet from the server if we are authenticated successfully. Otherwise we would get ERR packet.

picture6.png

  • 3 bytes are packet length
  • 1 byte is packet number
  • Affected rows – Int<1>
  • Server status – Int<2>
  • Warnings – Int<2>

That’s all. We have finished theory. Now it’s time to start practical part. On the second part of this article we will write our own MySQL native client from scratch using no external module or library

By October 2, 2020
Most popular
All articles