Or how I have learned to embrace http.RoundTripper

Using a http client in Go usually starts like this:

resp, err := http.Get("http://example.com/")

Everything works, until it doesn’t: a network blip, a connection reset, a slow
response, etc.

After some research (I recommend reading The complete guide to Go net/http
timeouts)
you may end up writing something similar to:

c := &http.Client{
    Transport: &http.Transport{
        Dial: (&net.Dialer{
            Timeout:   30 * time.Second,
            KeepAlive: 30 * time.Second,
        }).Dial,
        TLSHandshakeTimeout:   10 * time.Second,
        ResponseHeaderTimeout: 10 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
    },
}

req, _:= http.NewRequest(http.MethodGet, "https://www.google.com", nil)

resp, err := c.Do(req)

Ok! But now the question is: how does http.Client really work?

// http.Client - comments removed
type Client struct {
	Transport     RoundTripper
	CheckRedirect func(req *Request, via []*Request) error
	Jar           CookieJar
	Timeout       time.Duration
}

Ok… what is the RoundTripper?

type RoundTripper interface {
        RoundTrip(*Request) (*Response, error)
}

Way simpler than I would expect: for each request, it receives a response, if everything works.

The default transport.go
implementation is interesting. However, what else could I use RoundTripper
for?

Implementing a naïve Retry RoundTripper without any concurrency concerns or
request copy:

// Retry - http client with retry support
type Retry struct {
	http.RoundTripper
}

// Naive Retry - every 2 seconds
func (r *Retry) RoundTrip(req *http.Request) (*http.Response, error) {
	for {
		resp, err := r.RoundTripper.RoundTrip(req)

		// just an example
		// we potentially could retry on 429 for example
		if err == nil && resp.StatusCode < 500 {
			return resp, err
		}

		select {
		// check if canceled or timed-out
		case <-req.Context().Done():
			return resp, req.Context().Err()
		case <-time.After(2 * time.Second):
		}
	}
}

func main() {
	c := &http.Client{
		Transport: &Retry{http.DefaultTransport},
	}

	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://www.google.com", nil)
	if err != nil {
		log.Fatalf("Could not create request")
	}
	resp, err := c.Do(req)
	if err != nil {
		log.Fatalln(err)
	}
	defer resp.Body.Close()

	log.Println(resp)

}

The HTTP retry client continues retrying until it succeeds or the context
timeout reached.

A very simple http client cache:

type Cache struct {
	kv map[string]*bufio.Reader
	http.RoundTripper
}

func (c *Cache) RoundTrip(req *http.Request) (*http.Response, error) {
	if req.Method != http.MethodGet {
		return c.RoundTrip(req)
	}

	v, ok := c.kv[req.URL.String()]
	if ok {
		// return cached
		return http.ReadResponse(v, nil)

	}

	resp, err := c.RoundTripper.RoundTrip(req)
	if err != nil {
		return resp, err
	}

	body, err := httputil.DumpResponse(resp, true)
	if err != nil {
		return resp, err
	}

	r := bufio.NewReader(bytes.NewReader(body))
	c.kv[req.URL.String()] = r

	return resp, err
}

func main() {
	c := &http.Client{
		Transport: &Cache{
			RoundTripper: http.DefaultTransport,
			kv:           make(map[string]*bufio.Reader),
		},
	}

	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://www.google.com", nil)
	if err != nil {
		log.Fatalf("Could not create request")
	}
	resp, err := c.Do(req)
	if err != nil {
		log.Fatalln(err)
	}
	resp.Body.Close()

	log.Println(resp)
}

RoundTripper interface enables extending the HTTP behaviour in a composable way.

The more I learn about Go Standard Library, the more I acknowledge the attention
and craft put into it.