Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (Cyborg)
  • No Skin
Collapse
Brand Logo

CIRCLE WITH A DOT

  1. Home
  2. Uncategorized
  3. 83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

Scheduled Pinned Locked Moved Uncategorized
bbcwebstatspythonrequestsbricktop
11 Posts 6 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • tdp_org@mastodon.socialT This user is from outside of this forum
    tdp_org@mastodon.socialT This user is from outside of this forum
    tdp_org@mastodon.social
    wrote last edited by
    #1

    83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

    That's 84 million requests per day.

    In the words of Bricktop...

    #BBC #WebStats #PythonRequests #BrickTop

    Link Preview Image
    viss@mastodon.socialV relet@flipping.rocksR jamesog@mastodon.socialJ 3 Replies Last reply
    1
    0
    • tdp_org@mastodon.socialT tdp_org@mastodon.social

      83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

      That's 84 million requests per day.

      In the words of Bricktop...

      #BBC #WebStats #PythonRequests #BrickTop

      Link Preview Image
      viss@mastodon.socialV This user is from outside of this forum
      viss@mastodon.socialV This user is from outside of this forum
      viss@mastodon.social
      wrote last edited by
      #2

      @tdp_org have you checked out the source ips to see if its just a bunch of cloud vms or something?

      tdp_org@mastodon.socialT 1 Reply Last reply
      0
      • viss@mastodon.socialV viss@mastodon.social

        @tdp_org have you checked out the source ips to see if its just a bunch of cloud vms or something?

        tdp_org@mastodon.socialT This user is from outside of this forum
        tdp_org@mastodon.socialT This user is from outside of this forum
        tdp_org@mastodon.social
        wrote last edited by
        #3

        @Viss There's quarter of a million distinct IPs so no, I haven't done that but I have looked at the ASNs from which the requests come and they're mostly eyeball ISPs. I suspect there's some common router config or similar which uses it as "is the internet working" - the usual 😭🤣

        viss@mastodon.socialV 1 Reply Last reply
        0
        • tdp_org@mastodon.socialT tdp_org@mastodon.social

          @Viss There's quarter of a million distinct IPs so no, I haven't done that but I have looked at the ASNs from which the requests come and they're mostly eyeball ISPs. I suspect there's some common router config or similar which uses it as "is the internet working" - the usual 😭🤣

          viss@mastodon.socialV This user is from outside of this forum
          viss@mastodon.socialV This user is from outside of this forum
          viss@mastodon.social
          wrote last edited by
          #4

          @tdp_org ah, could possibly be botnet type stuff with all the edge devices that have been popped recently and turned into proxies?

          tdp_org@mastodon.socialT 1 Reply Last reply
          0
          • viss@mastodon.socialV viss@mastodon.social

            @tdp_org ah, could possibly be botnet type stuff with all the edge devices that have been popped recently and turned into proxies?

            tdp_org@mastodon.socialT This user is from outside of this forum
            tdp_org@mastodon.socialT This user is from outside of this forum
            tdp_org@mastodon.social
            wrote last edited by
            #5

            @Viss It's possible but it's so consistent that I doubt it.
            One of the many "joys" of being a relatively well-known presence on the internet I spose 🤣

            viss@mastodon.socialV simonzerafa@infosec.exchangeS doctormo@floss.socialD 3 Replies Last reply
            0
            • tdp_org@mastodon.socialT tdp_org@mastodon.social

              @Viss It's possible but it's so consistent that I doubt it.
              One of the many "joys" of being a relatively well-known presence on the internet I spose 🤣

              viss@mastodon.socialV This user is from outside of this forum
              viss@mastodon.socialV This user is from outside of this forum
              viss@mastodon.social
              wrote last edited by
              #6

              @tdp_org what are all those python clients actually requesting? is it just the same uri over and over again from 250k hosts?

              1 Reply Last reply
              0
              • tdp_org@mastodon.socialT tdp_org@mastodon.social

                @Viss It's possible but it's so consistent that I doubt it.
                One of the many "joys" of being a relatively well-known presence on the internet I spose 🤣

                simonzerafa@infosec.exchangeS This user is from outside of this forum
                simonzerafa@infosec.exchangeS This user is from outside of this forum
                simonzerafa@infosec.exchange
                wrote last edited by
                #7

                @tdp_org @Viss

                Maybe change the the domain to:

                BritishBroadcastingCorporation.co.uk

                So they have to take a long longer to type it into a script? 😉

                Any plans to block or otherwise deal with that traffic?

                1 Reply Last reply
                0
                • tdp_org@mastodon.socialT tdp_org@mastodon.social

                  83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

                  That's 84 million requests per day.

                  In the words of Bricktop...

                  #BBC #WebStats #PythonRequests #BrickTop

                  Link Preview Image
                  relet@flipping.rocksR This user is from outside of this forum
                  relet@flipping.rocksR This user is from outside of this forum
                  relet@flipping.rocks
                  wrote last edited by
                  #8

                  @tdp_org

                  What will be the impact on critical infrastructure if you decide to drop or reject all the connectivity checks from one day to another?

                  Care to find out?

                  1 Reply Last reply
                  0
                  • tdp_org@mastodon.socialT This user is from outside of this forum
                    tdp_org@mastodon.socialT This user is from outside of this forum
                    tdp_org@mastodon.social
                    wrote last edited by
                    #9

                    @Gaelan Yeah we're on plenty of those lists 🤣
                    I suppose it's a compliment of sorts

                    1 Reply Last reply
                    0
                    • tdp_org@mastodon.socialT tdp_org@mastodon.social

                      83% of requests (globally) to our apex homepages (bbc.co.uk/ and bbc.com/) are from "Python Requests".

                      That's 84 million requests per day.

                      In the words of Bricktop...

                      #BBC #WebStats #PythonRequests #BrickTop

                      Link Preview Image
                      jamesog@mastodon.socialJ This user is from outside of this forum
                      jamesog@mastodon.socialJ This user is from outside of this forum
                      jamesog@mastodon.social
                      wrote last edited by
                      #10

                      @tdp_org Default user agents are the bane of my life. I wish libraries would force users to set it correctly

                      1 Reply Last reply
                      0
                      • tdp_org@mastodon.socialT tdp_org@mastodon.social

                        @Viss It's possible but it's so consistent that I doubt it.
                        One of the many "joys" of being a relatively well-known presence on the internet I spose 🤣

                        doctormo@floss.socialD This user is from outside of this forum
                        doctormo@floss.socialD This user is from outside of this forum
                        doctormo@floss.social
                        wrote last edited by
                        #11

                        @tdp_org @Viss

                        Possible. I run inkscape's website and it's shocking what my ASN tools show. Some of the IPs slam requests, others hit with less frequency. But ASN plus the honey trap gets a decent set of addresses to block or degrade. (Our poor machine can't handle CDN levels of requests)

                        1 Reply Last reply
                        0
                        • R relay@relay.an.exchange shared this topic
                        Reply
                        • Reply as topic
                        Log in to reply
                        • Oldest to Newest
                        • Newest to Oldest
                        • Most Votes


                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • World
                        • Users
                        • Groups