0

I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 roads and 4 signals(agents).

  • 2
    What are you trying to observe? Markov models are powerful for making observation from a hidden variable. What are you starting states and what is the injected information? – JahKnows Jul 24 '17 at 14:04
  • 2
    I am trying to find a optimal policy for reducing the waiting time of traffic on signal. and about the sates I am confused i do not know how to start and proceed with multiple agents. –  Jul 24 '17 at 14:11
  • If you draw your intersection maybe we can get a better idea of what you mean. Is it a requirement to have multiple agents? If you can restrict that the all of the traffic lights are never set in such a way as to cause accidents (all green), it should reduce to a smaller state space and be easier to consider. – Jaden Travnik Aug 01 '17 at 01:55
  • As far i am able to think, you need traffic data for all roads, Based on that you can identify congestion time/traffic using MM/HMM/SSM. Based on traffic at each road at same time, you can optimise the waiting time. – Arpit Sisodia Aug 15 '17 at 05:44
  • it is ready now you can try it @ https://github.com/aawadall/SmartTrafficIntersection it is text based though, you can extend it to have a graphical component to it, or if you want you can port it to whatever language you prefer – A.Rashad Aug 28 '17 at 23:51

1 Answers1

2

Why not use a single agent to control the intersection with the following rules:

  • define 8 traffic lights
  • Each light has two possible values, 0 or 1, with 0 = Red and 1 = Green
  • Lights are as follows:
    • 0 = North to South
    • 1 = North to East (Left Turn)
    • 2 = East to West
    • 3 = East to South (Left Turn)
    • 4 = South to North
    • 5 = South to West (Left Turn)
    • 6 = West to East
    • 7 = West to North (Left Turn)
  • map lights into a bitmap of 8 bits, with the least significant bit (left bit) is the North to south

Given the above, we have the following valid moves, which are both State Space and Action Space :

  • State/Move 0 - Vertical Moves only = 10001000
  • State/Move 1 - Horizontal Moves only = 00100010
  • State/Move 2 - Originated from North only = 11000000
  • State/Move 3 - Originated from East only = 00110000
  • State/Move 4 - Originated from South only = 00001100
  • State/Move 5 - Originated from West only = 00000011
  • State/Move 6 - Vertical Diagonal only = 01000100
  • State/Move 7 - Horizontal Diagonal only = 00010001

i.e. S = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001}

and A = {10001000, 10001000, 00100010, 11000000, 00110000, 00001100, 00000011, 01000100, 00010001} enter image description here Now we have a ∈ A and s ∈ S and at any given s, a will cause the next state s' to have the same value of a

Final component to complete the moel, is the reward signal, which can be simply the negative sum of time spent per car in the intersection

i.e. rt+1 = -∑ci ∈ Cars time(ci)

a single agent, will be able to optimize the solution to avoid punishment (negative value caused by wait time)

Update

You can increase the state space to double or triple its size to accommodate for time, by appending a time counter component to the bitmap. in addition to an extra (9th) action of do nothing,

This should give a more subtle traffic controller. e.g. Vertical Traffic Only, at timer = 0 can be represented by 100010000000 or in hex 880, then at time 1 = 881, and so on until time 15 to be 88F.

Update 2

You can find the source code here and here

A.Rashad
  • 251
  • 1
  • 14
  • By the way, I started implementing this solution in C#, if you are interested, I can send you an invite to its git repository – A.Rashad Aug 24 '17 at 15:55